MaxIT

On launching into an e-learning course you leave AbilityLMS and enter the e-learning course. Said course is developed in an authoring tool (or set of program libraries) that is completely separate from our technology. The user experience in the course has nothing to do with how the AbilityLMS works. 

The course is the SENDER of tracking data to AbilityLMS. If the course does not send the data, AbilityLMS does not track, period. AbilityLMS is then the RECEIVER of data the course chooses to send. That raw data is logged for each user in each course. If a population of 1000 users took the course, there will be 1000 log records. Each user/course/tracking record has its own log. 

The history record in AbilitLMS links to the log file. By navigating to the history record from either the People/History tab or Course/History tab, you access the same log file as links from the History Log tab for a selected record. 

If there is no link as highlighted below, the course never spoke to the LMS. 

Let's be clear in what this means with a population of 100 users taking the course in question. 

If all 100% of the users show no SCORM API link in the above capture, then there is a problem in the course or in how the course is registered in the LMS. 

If you have 90 people who took the course and 10 users show no API log link, you have a problem in the user environment to sort out. 

The other link, the SCORM Log file link, is an inventory of launches, not tracking data. The SCORM API Log is a log of all course to LMS communication. This is the file you want to inspect to assess course communication behavior. 

As communication is logged, it is always appended to the log file. Scroll to the bottom to see the most recent communication. 

The course can GET information, called an LMSGetValue (1), from AbilityLMS and SET information, called LMSSetValue (2) into AbilityLMS as highlighted in the below picture. In (1), this is showing the course getting from AbilityLMS the Status the system for the user currently in the datavase. As a new launch, AbilityLMS will return "not attempted". 

In (2), you see the course setting in the AbilityLMS database the user is now "incomplete". When the AbilityLMS is asked to store or set data, the course will identify what field to update (Param1) and what value to park into the field (Param2). In most cases you are concerned with the LMSSetValue for "cmi.core.lesson_status". You want to see ultimately a PASS or COMPLETE status sent. if you do not see that in the log it means AbilityLMS never received a signal from the course to mark the user as Finished based on a Complete or Pass status. 

Scorm logs can be very large based on how the course is programmed, but the top to bottom flow of the log, where the most recent communication is at the bottom of the log makes it very easy to review the entire life cycle of the course to AbilityLMS communication. Once you learn the key things to look for, it can suggest where a course is breaking down or even suggest if the user lost connectivity. 

When inspecting the log, you will want to use the CTRL + F key to search for common values. Searching on "complete" or "pass" including the double quotes will tell you immediately if the course sent a Complete or Pass status. If AbilityLMS does not show finished on the history record for the user and you do see a "complete" or "pass" status in the log, then this is potentially an issue where a support ticket should be raised. 

Here are other common values the course might SET to the AbilityLMS

(1) cmi.core.score.raw represents the current score for the user if there is a quiz. As you scan the log and depending on the design of the course you will see the score increase. At the bottom of the log file is the most recent score the course reported. If you see the user has reached what you believe is the mastery score but they do not show as Finished in AbilityLMS, navigate to the course record, course modules tab and check the mastery score is at the desired passing value. 

(2), (3) cmi.core.score.max and cmi.core.score.min represent the highest you can score and lowest. it is rarely used or is relevant in course design. 

(4) cmi.core.lesson_location typically represents a bookmark position in the course. In this case, the user exited on what is probably the 4th slide or page in the lesson. The values and lingo is determined by the course. Not all courses will use this field, but it can be helpful particularly if there is a course freeze to identify the problem page in the course. 

(5) cmi.core.supend_date represents the work you have done in the course. What data goes here has no structure as far as AbilityLMS is concerned and represents whatever data in whatever structure the course sends. In most cases this is jibberish to everyone but the course engine, but over time you may start to see patterns. For example, some courses will represent pages in the course as 0s if not visited and 1s if visited. A 10 page course then initially will be 0000000000 and if the user has visited 5 pages you would see 1111100000 and if all pages are visited 1111111111. It can be useful to compare this value for a user who has completed the course to someone who has not. Like lesson location it can suggest a problem position in the course. 

Looking for the "cmi.core.exit" will tell you if the user exited the course in an order fashion. If you do not see that (towards the end of the log file), it strongly suggests they lost internet connectivity or abruptly closed the browser in such a way to not allow the course to communicate to the LMS. 

The Mastery Score Problem 

When a course is registered into AbilityLMS, one of the set-up properties for the course is Mastery Score. This score represents the passing score of the course. The course will ask for mastery score with a GET command and will use that score to determine if a user has passed a quiz. This design has weaknesses. 

You have to pay attention to the mastery score setting when publishing the course. If it is set to 100% and loaded into AbilityLMS and then you make a change in the authoring tool and republish only the content to the server, AbilityLMS is not going to know of the change and you have to manually edit the Course Module record to set the right score. 

Many course developers do not understand the significance in the difference between Complete and Pass. In SCORM-speak, mastery score should only come in play when the course is sending a Pass status, and a Complete status should ignore the mastery score. In the real world, we have too many clients who published 50+ courses only to realize the Complete status was used and they start seeing tracking in AbilityLMS where the user's score is less than the a passing score. When they realize they have to republish their course library and time commitment, there is too much pain and it is easier if we just change the behavior of AbilityLMS to treat a Complete status with the presence of a mastery score the same way as a course using a Pass status, so the pain of republishing and reloading is entirely avoided. If you want a course to not consider mastery score with a Complete status (eg default behavior of how SCORM should work when the course sends a Complete status), simply set the mastery score in the course module record to 0 if it is not set already. In most cases, this is exactly how it is set-up for default SCORM behavior when a course is loaded into AbilityLMS. 

The Old Course / New Course problem

Along comes a course update and all the sudden tracking in the course breaks. This might happen for users who are in-progress in a course and the new course changes how the course internally determines completion. Since AbilityLMS is already tracking based on the old course structure, these users never get to a complete status. Normally this is reflected in the suspend data field in the log. In any event, when uploading a new version to the course, you should consider cancelling all users in-progress so they start over. You should also consider publishing the new version of the course to a new folder just to eliminate an caching effects where the old version of the course is loaded to the user's browser. It is also a good idea to put a market in the course like a version number to make sure you uploaded the right version of the course to the server. 

It works in SCORM Cloud means very little. 

It is the responsibility of the course to SEND the tracking data. Course and Tool vendors tend to not have a reliable way to tell if the course is sending the data or not, so they will say load the course in SCORM Cloud or some other LMS to see what the results. Often times the course will work fine and the question is asked if it works there and not in AbilityLMS, the problem must be in AbilityLMS. 

This is a false analysis and has to be challenged. 

Again, if you see no API log link as described above for any user, you have a problem in the course or how the course is set-up in the LMS. For example, you get a course from a vendor and they give you a SCORM2004 course and not SCORM 1.2 version. We can help in troubleshooting this sort of issue. 

If you see the API log, and it is not tracking to finished to AbilityLMS, it means the course is speaking to LMS. More often then not a user completing the course under the same controlled tolerances as the 3rd party system will produce the same results. 

Best Practices to Consider

We can offer 4 best practices you should ask of all users. Think of these best practices as e-learning hygiene. A useful comparison is a person who does not have good dental hygiene. Over time their teeth deteriorate and their breath stinks. Likewise, we don't want taking e-learning and supporting e-learning to become a more painful experience than it needs to be. Best practices will keep both user and support rep from having to endure that stinky experience with an ounce of prevention. 

(1) Before starting any e-learning close all applications including all browsers and open a new browser window. The only application running should be the browser and the only site should be AbilityLMS. When you do not have 50 browser tabs open and 30 other applications, the workstation is not so stressed. 

(2) Do your training in one session. If you get distracted with a phone call or some other activity and you are away from the workstation for more than 30 minutes, follow rule 1. We see all the time in our logs where users we're in a learning course for 48 or more hours. 

(3) Do not spend more than 60 minutes in a course at 1 time. Give your brain a rest. 

(4) Do not rush through the course. Many courses will track that all pages are visited and a user quick on the click can easily break the internal tracking of what the course is sending to the LMS. Remember, the LMS is only the RECEIVER and the course, eg the SENDER, must send good data. 

If you will follow these best practices, not only do you drastically shrink the range of causes for courses not to communicate to AbilityLMS, you will get better results in terms of the learning/training that takes place because you eliminate distractions. Sure there is probably some happy medium in these best practices, but the moment you start having problems with a user/course, you should ask for this behavior because you eliminate many (not all) environment issues that might cause the course to misbehave and can more easily zero in on other problems.