A user sits down to do their e-learning course. They judiciously complete all exercises and review all lessons in the course. They get to the end of the course and see a Finished message. They close the course window, the LMS window refreshes and says In-Progress. You get the call the LMS is broken.
Or maybe the user starts a courses and leaves midway through. They come back later only to see they are sent back to the start of the course. You get the call the LMS is broken.
If you ever had these calls or calls about courseware problems, this guide is for you. This guide offers advice in how to troubleshoot tracking problems in a course. Please be sure to read the other courseware articles as some of the concepts presented here are more fully discussed in these other articles.
There are two truths you must consider in troubleshooting courseware. Truth One is the LMS stores the tracking data the course communicates. Truth Two, and the truth most people do not fully appreciate, the course has responsibility to communicated to the LMS. The LMS cannot make the course communicate.
Courses are very much like inmates in a US prison. The inmate can make calls to an approved phone number. A course can make calls to the LMS. An inmate cannot receive incoming calls. An LMS cannot go and get data from the course. Going back to the opening sentence, so while the user may insist the have completed the course, how do you know the course posted back to the LMS? Well an inmate has the entire conversation recorded in case they incriminate themselves in some other crime, and we do the same in AbilityLMS, recording each and every request from the course to the LMS to an audit log. That is an excellent start and covers most use cases, but there are layers here to unravel.
This audit log is authoritative. There will be no tracking in AbilityLMS if it is also not in the log which makes this an incontestable source of truth for truth one. Further, each history record has its own log record and a user that completes the course over several sessions, has the same log file appended with whatever the courses communicates offering a complete life-cycle of the user activity in the course. The log file is available for System Administrators to view when looking at training tracking for an AICC or SCORM course. Again, the course has responsibility to make the call to the LMS. The LMS cannot magically extract data from the course.
One last point on the course communication auditing built into AbilityLMS is the log capture occurs BEFORE the system ever tries to read the data the course is sending. If we get bad data from the course, AbilityLMS may not be able to process the data, but it will show in the log. If you need to inspect the data the course gets when it resumes, you can inspect that too. This last point can be helpful as sometimes a course will post bad data we just park to the history record. We cannot regulate that data. If the course posts bad data, the user resumes, the course gets the bad tracking data back. Regardless the audit log is you first area to inspect if there is a tracking problem or the course is launching but misbehaving.
We have Truth One covered.
Administrators can inspect the log easily. Look up a user in the People listing, click the history tab (1), click the green "i" icon (2) for a history record and then History Log (3) tab in the new popup window. You can do the same from Manager Menu, Courses screen where you look up a course, then in the history record the person who has history for the course. Repeat the above steps.
If an AICC course, you will see a single link. If you see no links it means the course never posted to AbilityLMS. Assuming something was posted, click the link to view the AICC log.
Each log link bring up a unique log for the tracking record for the user + course. If the user launches the course 10 times, the same log file is updated, just appended for tracking data for subsequent launches.
Once in the log, you will see a lot of text. The log file is always appended as the Learner progresses through the course, so scroll to the bottom to see the most recent course-to-LMS communication that was logged. You can also press the CTRL + F keys to perform a search on either the phrase - lesson_status="passed" or lesson_status="complete" - and if you see that in the log file, then it means the course posted a pass or complete status to the LMS. We would expect then to see a Finished status on the history record unless you have workflows, such as an Assessment or Exam, that might require some other action before the history is updated to "Finished". Most e-learning courses do not have workflows.
If the course is a SCORM course, you will see two links. Click on the API link (1). SCORM has more moving parts, but the important link is the "API log file" link. If you do not see a link for the API log, then the course is not communicating to the LMS.
Press the CTRL + F keys and search for Param2="Completed" or Param2="Pass" to see if the course ever posted a Complete or Pass status. You can also scroll to the end of the log file to see the most recent completions.
Please note the log file is only available for SYSTEM ADMIN roles. In some environments the History log tab is not enabled. Please submit a support ticket if you do not see the tab. The tab will only appear for AICC and SCORM courses.
If the LMS is not showing finished, and after inspecting the audit log and the audit log does not shows a COMPLETE or PASS status, then your problem solving has to focus on the second truth as to what the course is communicating. Let's be crystal clear. when problem solving in the realm of the second truth, you are supporting the operation of course, not the LMS.
The fundamental question is how do you know the course is properly communicating?
Often, you call the course vendor or tool vendor or google troubleshooting techniques. Most of the time they will reply that they can validate expected behavior on a 3rd party site like ScormCloud or in another LMS. That's great in that in confirms the course will track properly, and more than likely you will get the same results in our Sandbox environment, because these are all controlled environments. The real world is what is happening in the course, on the local workstation and in the network. It bears repeating, how do you know the course is properly communicating?
And here is where it can become difficult because the course vendor / tool vendor will often say if it works elsewhere or it pasts the conformance tests, it is not their problem. In essence, we are saying that if expected tracking is not in our authoritative and incontestable audit log, we are not getting the tracking it is not our problem. The client is stuck in the middle so how do you make progress. Well let's cut to the chase that posting in a controlled environment is a logical fallacy of equivalents and problem solving has to be around validating the course is posting properly in your production environment. The course vendor must be held accountable to prove the course is posting in the production environment. They may well be right the course is posting properly and something outside of their and our control is preventing the LMS from getting the data, but you cannot problem solve until you can actually see the course posting data much like you can see our system processing incoming posts from the course.
Course engines are designed to work in known environments and generally do a good job, but this does not mean there might be some error in how the course was published, some use case while a user is in the course, or some local use case the course engine is not able to handle in your environment. If not a problem in the course, the environmental problems are numerous. Some examples include Anti-Virus pulling the workstation to 100% on a scan, proxy server caching / filtering on a trusted network you do not get when gong to the outside www, packet prioritization, network outage.... and the list goes on and on. It bears repeating, how do you the course is properly communicating?
Maybe it is a course problem, maybe it is not. If expected communication is not showing in our logs, the LMS is not getting the data. To effectively troubleshoot you have to be able to inspect the course actually posting in your production environment. You cannot assume because it does so in a controlled environment it does so in your production environment. If we do not see it in our log, we will must ask you demonstrate the course actually posting in your environment or we may ask support be covered under a statement of work with professional services charges to cover the labor of supporting technology we did not develop and/or some weird way your network is causing courseware not to post to the LMS.
One excellent self-service approach that will work if the course is communicating from the user's workstation directly to the LMS (the way it works most of the time) and not through a back-end server to server post (which is the case for SkillSoft which never has a tracking problem) is to use a web debug proxy. A free open-source web debug proxy is called Fiddler which you can download. See this KB article for more information: https://abilitylms.freshdesk.com/support/solutions/articles/33000097122-using-fiddler-to-help-solve-courseware-problems. Fiddler can be installed to the problem user's workstation and while the user is going through the course can be configured to record the web traffic to the LMS. This will provide incontestable data of what data came in and what data came out. The recording can be shared with the course / tool vendor and MaxIT and it can save a lot of time. This will tell you what the course is communicating.
One thing about Fiddler, if you have Flash-based courses and proxy servers that control access to the outside web wired into the browser, Fiddler will bypass the proxy server and problem courses often magically start working. Again, our audit logs are incontestable and if this happen to you, it means you have proxy servers doing weird caching. The course vendor might have technology to get around the caching as we know of one vendor actively trying to solve the problem. We recommend you move away from Flash-based courses. https://abilitylms.freshdesk.com/support/solutions/articles/33000211017-end-of-flash
Let's summarize. Truth one is our responsibility and we have you covered. Truth two is only our responsibility if you subscribe to our courses. If you subscribe to other course vendors or build your own content, our logging tells us and you what the course is sending. If the course is not committing good data, you need to engage the vendors producing that technology to sort that out before reaching out to MaxIT for assistance.
Here are some other tips to help troubleshoot:
See if this is a long-running problem or a new problem. Go to the Course Menu and look up the course. In the history tab, sort in descending order on status date. If you see 100% In-Progress, the course never worked. If you see In-Progress for more recent date ranges, perhaps some update to the course caused it to stop working.
If you see a mix of users who have completed and users who have not in a date range, then your support does becomes more of a headache. Is there some use case in the course breaking communication, or is there some problem in the user environment? Please understand, we support what is in the log. MaxIT cannot be asked to support a problem in the course posting to the LMS bad data or in the design of the course.
Often we see a problem when a course is updated or republished to the server where content is cached. This is a problem in the user environment. The user gets the cached version and not the most recent version. If you are building courses it is a recommended best practice to put a version number or some other marker only you will see to indicate this is the latest versions of the course. It stinks to burn hours tracking down a problem only to realize the user is trying to run a cached version of the course.
Clearing history can help. If a flash based course, you have browser history, browser local storage and flash cache to clear. You will need to google how to clear browser local storage and flash cache. Simply clearing browser history is not enough for flash-based courses. Flash in general has become a big headache. Browsers are making it harder to run flash, and the many security updates to flash, the nature of proxy servers and the many updates to Microsoft Internet Explorer and Edge in particular make support for flash dicey. If you can force the issue to support only HTML-5 courses you should. In 2020, flash as a technology is officially no longer supported. See: https://abilitylms.freshdesk.com/support/solutions/articles/33000211017-end-of-flash
If you hold CTRL + F5, you can force a hard refresh of your browser window, and this can often help for cache problems. You may also want to try the course running in privacy mode (IE/Edge) or incognito mode (chrome) or run entirely in another browser. What these tips do is help you qualify if the problem in a course is local to a browser or not.
If you have access to where the web server stores files on the server via FTP or some other means of connection, make a small change to the root folder name of the course and have that name reflected in the launch URL in the course modules tab of the LMS. If the course is published to C:\clients\YourCompany\Training\CBT100 and the launch URL is http://YourCompany.learnerhall.com/training/CBT100, if you edit CBT100 on both the server and course module record to something like CBT100_2, this will force the workstation to load a new version of the course. This will not solve caching problems but it will force the user to load a fresh version of the course.
If you can, put the course into debug mode so you can see the course actually making requests for data or trying to commit data to AbilityLMS. Most of the modern course engines support a debug mode, but you need to engage them directly. Courses have their own design patterns; hence their own bugs to sort through. At the end of the day the course and the environment has to be able to commit data to AbilityLMS. We can tell if AbilityLMS got the data, the course or tool vendor has to prove to you it actually posted to the LMS. Just seeing a Complete message on the course screen does not mean the course actually communicated. If it did communicate, it will be in the logs and should of course be reflected in the system with a status of finished.
When running a course, sometimes it will break or freeze mid-stream. One trick is when in the course press the F12 function key, then the console tab. Look or JS errors or 404 errors. Remember, on launch you are not in the LMS, you are in the course. Perhaps republishing the course will fix the problem. Perhaps the course needs some MIME type on the web server. Your course / tool vendor should be able to diagnose any messages in the console tab.
To recap, when courseware decides to be a problem, your first look should be in our log. It does not matter if the user sees a Complete or Pass status in the course; it matters if the course posted a Complete or Pass status to the LMS. If it did, it must show up in our log. Only on the most rare occasions does a course posting not make it to the history record, but it must always, 100% of the time show in the audit log. AbilityLMS cannot make the course communicate. If you need to engage the authoring tool or content vendor, you should let them know that all courseware to LMS activity is logged as this can help hold them accountable. If our logs show no tracking is posted by the course, they have to prove to you the course is posting the data. That is works in other environments is only proving it is posting in a controlled environment and that is not an acceptable answer to maxIT. Fiddler can be a useful neutral 3rd party, but the content vendor has to prove it is posting and this must be communicated in clear terms to the content or tool vendor. MaxIT can support only what is is getting.
If a flash based course running to an internal work network is causing problems, you likely will have to engage your network resources to assist in proxy server changes as the course vendor may have never seen the effect. We cannot tell you how to fix that problem because we no longer engage the client on anything flash based.
No vendor is perfect including MaxIT. We are always happy to participate in calls with content vendors when troubleshooting is needed; however, it is asked you have reviewed the logs and the content vendor has data to support it is posting to AbilityLMS well formed data, not posting to SCORMCLOUD or some other environment. Feel free to share this KB article with them as it can be useful to setting expectations that the focus should be on solving your tracking problems, not placing wasting time or shifting responsibility.
MaxIT support starts at getting good data to our logs.