Lesson Attendance Optimisation

WORK IN PROGRESS - MAY CONTAIN INCORRECT ADVICE

In a similar way to the work on optimising a cache of session marks, this document aims to explore efficient ways to update a cache of lesson marks.  

This particular area is an obvious large volume of data with frequent change which would in theory offer most benefit from optimisation and potentially the highest cost of operating in terms of both CPU usage and egress.  As an estimator:

• 1000 pupil school
• 40 periods per week
• 50 weeks per year (ease of maths and non teaching weeks may also have a mark)
• 2 Million marks per year.

The worst extraction model would be to extract every mark every say 6 minutes (as a comparison with session attendance)  and pulling 240x2M marks ~ 1/2B marks per day is clearly a non-starter because on average in our example 8-10K marks change each day.

As a salient additional piece of guidance, every call made to hosted/connected systems has an additional overhead estimated presently to be in the order of 3s. Hence the design of the solution ideally needs to minimise the number of server calls,

As an example if calls to get lesson data were available:

• By teacher (estimate 67 per school)
• Per period (40 periods per week x 50 weeks = 2000)
• Per student (1000)
• Per day (365)
• By report (1)

2000 calls for example would take the SQL time plus 2000x3s = 100 minutes.  An additional 3s for a report or partner API call is probably noise in the wider integration.

The first optimisation is probably to restrict the time window of the data store.  In most cases, the current academic year is likely to be the scope of interest.  If this is not the case, the initial advice would be to export previous years data as a static cache and avoid updating it if at all possible. 

The lesson mark APIs work well when you need marks for a class for a day because it is simple enough to get the lessons taught by a teacher on any given day and then to invoke a call to get the marks for the lesson. However if you want to get marks for a year then in theory:

• Pupils come and go from the school
• Staff come and go from the school
• Lessons may be fixed for the year or they may not
• Even then timetables can be change at any time.

So to build the cache we need to call the partner API as few times as possible but avoiding huge payloads.  The first effort was to iterate through year groups using the same model as for session attendance however:

string test = attendanceRead.GetXmlLessonAttendancesExtended(personid, 0, start, end); //
XmlDocument getXmlLessonAttExtMultipleTeachers_ByGrp = attendanceRead.GetXmlLessonAttendancesExtendedMultipleTeachers(0, groupid, start, end);   

either of the above return nothing because it expects a teaching group, year groups are not usually timetabled as a teaching group which explains the lack of lesson marks for a year group.  An alternative mechanism that gets all of the teaching groups for the period in question (usually an academic year) is therefore needed.

Option #1: Curriculum cache

foreach (int groupId in SIMSInterface.Lesson.GroupsIds)
{
       XmlDocument d = SIMSInterface.Lesson.BaseLessonLoad(groupId,StartOfTime, t1);
}
--Where
partial class Lesson
{
        private static List<int> _GroupIds = new List<int>();
        public static List<int> GroupsIds 
        {
            get 
            { 
                if (_GroupIds.Count == 0)
                {
                    GroupLoad();
                }
                return _GroupIds; 
            }
        }
        private static void GroupLoad()
        {
            SIMS.Processes.CurrCache.Populate();

            for (int g = 0; g < SIMS.Entities.CurrCache.Curriculum.GroupCount; g++)
            {
                CurrGroup grp = SIMS.Entities.CurrCache.Curriculum.Group(g);
                _GroupIds.Add(grp.GroupID);
            }
        }
}

This uses the curriculum cache in SIMS which is expected to contain the list of classes in the current academic year. This mechanism may be problematic for earlier academic years