Here we discuss the current global market for online proctoring – also known as remote invigilation – and its future impact on elearning and computer based assessment.
We believe this will be a $10 Billion market by 2026
Background
Examinations, assessment and testing is a huge operation in the UK, US and worldwide. From schools with GCSE, A-level, K-12 and baccalaureate examinations, to driving-theory test-centres, to universities, thousands of exams are taken every day.
Included in this operation are a multitude of processes including certification, marking, invigilation and standardisation. This means that examination is not only a huge operation, but it is a huge potential market to those who can bring innovation and cost-savings to the examination operation. As examination operations involve so many processes, it seems that there is indeed potential for cost and time-saving innovations.
The most notable time and cost-saving innovation we have seen in examinations recently is the move towards computer-based testing. Computer-based testing, where a candidate takes a test or exam on a computer rather than by using pen-and-paper brings to examination the benefits of digitisation, such as faster operations, less physical material, and the opportunity for more testing locations.
Assessments of knowledge and competences are (our research tells us) likely to grow to reflect the changing nature of the workplace.
The Assessment Markets
The global market
The global computer based assessment market is worth an estimated $42 billion. It is forecast to grow very strongly due to the increasing internationalisation of education, increasing MOOC usage, and the demand for certification and regular re-certification from an increasing number of professionals.
We see remote invigilation – live, online identification and monitoring of the candidate taking an exam – as the next step for computer based assessment. Both in terms of technology and convenience, we believe this $42 billion market is ready to be tapped by remote invigilation.
Let’s put that number in context. In 2013 Learning Light worked with IBIS Capital to produce what is still viewed to be one of the most informative pieces of research into Edtech and elearning. In 2013 we viewed the global elearning market as $91 billion in 2012. Computer-based, locally-invigilated testing is equivalent to 46% of the elearning market at that time.
We see this cost as a major break on the growth of the e-learning market. As the cost of building assessments on e-learning courses and MOOCs is much less than the creation of engaging effective learning materials, but the cost of identity management and invigilation is considerable indeed.
But all this assessment comes at a cost, and when interviewed by Education Investor about Digital Assess, the author said this:
“The transition in the workplace from the tyranny of continuing professional development to the demonstration of actual competence is happening, and happening now. I believe that this transition will continue and assessing competence in the workplace effectively and designing out the burden of paper-based assessment systems will be hugely exciting, as, in my view, assessment costs are out of proportion to training delivery costs now.”
Computer-based testing
Computer-based testing is similar to pen-and-paper testing in many ways as, in its most basic form, the exam paper has simply been replaced with a computer. The most important similarities to note for this report are that the candidate must still take the test at a certain location and at a certain time, and that the exam, if it is to be conducted securely, must still be invigilated by a real person.
Computer-based testing in its most basic form can take place in a school or a university, but we have recently seen the growth of specialised testing centres, complete with an invigilator and often with PCs for candidates. The most prominent example of such a service is Pearson Vue.
Created in 2003, the Pearson Vue service enables anyone with a sufficient location to register to become a Pearson Vue test centre. Over 500 different organisations across the world have their certifications managed and delivered by Pearson Vue, ranging from American Academy of Implant Dentistry (AAID) and The Royal Australian and New Zealand College of Psychiatrists (RANZCP) to Wisconsin Barbers and Cosmetologists and the UK awarding body WAMITAB.
Pearson Vue reports delivering 13 million tests in the USA. This figure grew by 9% in their 2014 annual report and the numbers are mostly driven mainly by IT qualifications, State regulatory qualifications and professional certifications. However, Pearson Vue is constantly expanding and is moving in to the MOOC market.
The FutureLearn MOOC from the Open University will use Pearson Vue’s global network of secure testing centres to deliver 1 million statements of attainment through an exam.
In 2015 Pearson’s annual report showed Pearson Vue administering 14,200,000 tests and Versant PTE delivering 1,058,000 (up from 827,000) in 2014. These are tests in the Professional market and not in the education sector.
Assessment services (which includes education) represents 29% of Pearson’s business.
While there are complications around the two dominant players of CBT in the UK, Pearson Vue and LearnDirect we know from LearnDirect’s most recent accounts that CBT was worth £10.3 million in FY 15, up from £8.0 million in FY 14. This was for 2 government contracts and excluded the UK’s driving test theory, which Learndirect is still (possibly) slated to deliver, though it did sell this contract to Pearson Vue prior to the Competition and Markets authority stepping in.
The CertiPort career-oriented certification solutions that prepare individuals with the digital skills and credentials from Pearson uses 12,000 authorized testing centres globally and delivers 3 million certification exams annually.
ETS (Education Testing Services) is a US giant of testing, with a reported $1,138,103,154 turnover in 2013 further indicates the overall size of the market.
Prometric (part of ETS) operates 8000 test centres in 160 countries. Prometric offers testing for Academic, Corporate, Government, Financial, Healthcare, Information Technology and Professional Association, examining about 9 million students per year across the globe.
In the UK the DVLA contract holder to be decided, (but presently Pearson) will deliver 1.7 million tests per annum.
QuestionMark reports having 15,000 customers and it delivers 20 million tests per annum.
Yardstick, based in Canada has helped over 12 million learners “achieve competency” principally by assessment since inception in 2005.
These tests – many many millions of tests – will have been invigilated in some form or another.
Test centres today
A test centre enables computer-based testing to take place in a secure environment, where the location is strictly controlled and candidates are monitored by an invigilator. This situation enables the two essential requirements of a secure examination to take place: validation of candidate identity and prevention of cheating.
These two essential processes are referred to as identity verification and invigilation.
Computer-based testing centres save time and money compared to pen-and-paper examinations by digitising exam papers and marking, and by enabling candidates to take their test in a variety of locations. It seems that further costs and time could be saved if there were no locational requirements for the candidates, so a candidate could take their exam anywhere, eliminating travel and reducing time requirements, and if a human invigilator was not required to ensure the security of the exam.
However, there will always be a locational requirement with a test centre, and some form of invigilation will always be required to ensure identity verification and to detect and deter cheating. To save the additional time and money, then we need an examination solution that has no locational requirements, but can maintain security through identity verification and invigilation.
This is where remote invigilation (or proctoring) comes in. Remote invigilation removes the need for an exam to take place in a certain place, as an exam can be taken at any appropriate computer, and it can be proctored automatically/remotely without the presence of a person required.
Remote invigilation
Remote proctoring is becoming a well-established service in the US. The US market leaders are Software Secure, who have been established for 15 years; Kryterion; Proctor Cam, who were bought by Pearson in 2015; Examity, and ProctorU – who have been in business for over a decade and benefit from partnerships with a number of US colleges and universities.
It is interesting to look at these US vendors, particularly ProctorU, to see how they promote their products. Both vendors focus heavily on the use and the promotion of the term ‘proctoring’ and, in particular, ‘online proctoring’. ‘Online proctoring’ receives over 4 times more monthly searches than ‘remote proctoring’.
Furthermore, searches for online proctoring have seen a general growth in recent years:
The term ‘proctoring’ is well established in the US and is considered synonymous with our use of ‘invigilation’. However, ‘proctoring’ is not a well-established term, at least not to mean ‘invigilation’, in the UK. For example, none of the Learning Light team knew the meaning of ‘proctoring’ before it was explained to them.
What’s more, according to vendors of online assessment and invigilation software in the UK, ‘proctoring’ means different things outside of the US than it does to North Americans. For example, in Britain a proctor is often seen as an officer at a university; in Scotland, a proctor is someone who represents the Clergy, while in both the UK and Australia, a proctor has historically been a solicitor or other type of legal professional.
These examples show that ‘proctoring’ is not a well-established term outside the US, and that in particular it is not (yet at least) seen as synonymous with ‘invigilation’. It has been suggested by other leading UK vendors that, if one wishes to target the UK market for online assessment services, that the term ‘invigilation’ is used instead of ‘proctoring’, as this term is well recognised and accepted.
So there is clear requirement to broker some common understanding of the term, and as we are a UK-based organisation, we will use the term invigilation.
The main drivers for growth
So what do we expect to drive growth in the remote invigilation market?
We can easily assume Remote invigilation as a function of computer based assessment.
Of course Remote invigilation will also design out costs of secure assessment centres quite significantly, as well as saving learners time and money. This will be a powerful driver by awarding organisations and by learners voting with their feet, (or actually not having to use their feet!).
However, it will broaden into other areas as it is not merely a substitute for secure test centres delivering computer based assessments and tests, but provide a massive enabling change to education globally going beyond disruptive to secure assessment services.
Remote proctoring vendors have made claims of up to 25% savings on examination costs in education.
If we step into Universities and Colleges of Education, exam season puts huge logistical strain on these organisations for invigilating students with sports halls, dance halls and other ancillary facilities all temporarily becoming examination centres.
Many exams revert to pen and paper (or easily marked paper based multiple choice questions) for examinations as there are too few ‘smart exam desks’ to use. Additionally there are not enough computer labs to administer exam condition environments.
The breadth of exam activities taking place in these centres are huge as are the numbers or learners attending the test centres, all of which necessitates complex scheduling leading to the ever-growing costs of assessment and testing for many organisations.
The global computer-based testing market is one of those background services that attract little attention in the world of edtech, but it is a huge market that we believe is about to grow massively.
Recently, the computer-based testing market has been subject to some exciting innovation and disruption. While technology has enabled online assessments to take place wherever there is an internet connection, the challenge of ensuring the veracity of the remote examinee is still a challenge.
The above situations are all example of computer-based invigilation. The move from pen and paper examinations to computer-based examinations has allowed to exams to take place in a variety of locations, bringing with it the benefits of saving time, and of convenience. However, remote invigilation is the next step in this evolution, as it removes the requirement for exams to take place in a certain location.
Other drivers
In the UK the increasing focus on work based apprenticeships and the alignment of learning and assessments to actual organisational needs is likely to stimulate demand for remote assessment of practical tasks in the workplace, not the test centre.
Typically assessors will undertake work based assessments, but our research indicates assessment costs represent a growing cost of delivery and will soon exceed 30% of the cost
An interesting generation of new assessment tools are emerging to support remote assessing in the workplace. For example, MyKnowledgeMap (MKM) has been innovative in this field with its Myprogress.
The ever more sophisticated on-line examinations produced by BTL with its Surpass Assessment platform will also benefit from online invigilation and further create the virtuous circle of making learning and assessment accessible worldwide.
The shape of the market – US
We assume its market is both educational, vocational and in the corporate learning and training market. We have identified the huge potential of remote invigilation to unlock the potential of e-learning and assessments in many disciplines where a robust assessment of an individual’s skills and learning is required.
The opportunity around MOOCs is potentially huge where a desire to make money from participation and certification is important. Though we note from our observations a slowing of the free MOOC movement and detect a move to SPOCs in HE at present.
The opportunity is emerging for colleges and universities to maximise their distance learning offer as education internationalises. With estimates that 80% of all university students will be exposed to and use online learning during their studies, the use of online assessments and online remote invigilation can only grow.
The opportunity to in addition re-model how universities deliver exams utilising their fixed building infrastructures is an added bonus.
The US market is moving, but still immature, but we anticipate a rapid maturing in the education sector, particularly in the middle range and community college markets where learning models are changing and what the Gates Foundation refers to the un-conventional student is becoming the new norm.
In the corporate market the trend is to utilise RP-I for product certification training with companies and industry bodies and associations using ProctorU.
ProctorU is working with 450 institutions including Realtors (estate agents), ProctorU is internationalising as well, the British promotional merchandise association is a user. Integration with LMS such as Blackboard and others as well as with question bank provider Question Mark is again a feature of rapid uptake potential.
ProctorU offers services that involve human proctors delivering the actual session, operating from call centres in the USA. However, they also have a fully-automated service line that uses highly advanced, artificial intelligence technologies.
New technology that uses a lightweight headset with a 360 degree camera has been developed by Proctor360, and this could be a game-changer for the market. The headset can be mailed out to learners for their examinations, then returned for refurbishment and mailing out to the next student.
The company is run by Ganga Bathula, who has worked in the brick and mortar examinations for over a decade, and has the founder and former CEO of ProctorU, Don Kassner among its team of industry experts, so it will be interesting to see how they change the market with extra layers of security that their technology promises.
We estimate the US remote invigilation market is worth in the region of $60 million per annum at present. We see a market growing rapidly and while it will eat into the revenues of the established industry (or as is happening the established players buy up this industry) it will grow into new market segments.
Price points for remote assessment will quickly fall as operations scale and technologies emerge that drive out people in the operation.
The shape of the market – UK
The UK and European markets are yet to ignite, but the old prophecy of what happens in the US usually arrives firstly in the UK and then into Europe.
With SMOWL, TestReach and Comprobo active in the market and several universities using Skype and expressing interest in remote invigilation the market looks set to mature fast.
Awarding bodies will need convincing, but unique UK market drivers such as the FELTAG requirements and massive downward funding pressure on further education provision and the move to work based learning (with apprenticeships) the drive for flexible and secure testing in the workplace will grow and grow.
In the UK, awarding body regulations organisations do not see an issue with using remote invigilation services, and are most interested in maintaining rigorous processes and seeing that protocols are followed.
Professional bodies, such as AQA and OCR, who provide invigilation services often don’t have the physical structure available to invigilate exams, and to provide this would be a large cost to them. These bodies have good brand names, and can use this to leverage overseas expansion. Online invigilation services are therefore of great interest them, as they can enable them to expand whilst minimising cost.
Our research has shown that awarding bodies and organisations, colleges and universities, and businesses are generally positive towards the adoption of online invigilation.
It has been suggested that, in the UK, we are seeing steady growth with regards to the adoption of online invigilation, and that the tipping point on this scales with occur within the next 2-5 years, prompting a large increase in demand for online invigilation services. In comparison to the US market, the UK is as usual following in their footsteps, where the online proctoring market has already seen this large increase in demand, and has the market has levelled out to become well established.
We see the global growth in adoption of remote invigilation solutions to be huge as not only will the demand for trust in the credibility of online certifications be increasing, but this is compounded by the fact that it sits within the fast growing wider market of elearning as a whole.
The Existing Remote Invigilation Situation
We have recently seen home-made attempts at remote invigilation, but these attempts are not based upon the application of a robust system built for purpose. For example, replicating University Exams for special circumstances, such as when the examinee is unavoidably away from campus, or has complex logistical needs, has seen Skype-based invigilation emerge in certain Universities on an ad hoc basis. In these cases, the tutor often undertakes the invigilation. This is clearly not ideal and the university remote examination process, even in topics such as computing, is out of touch with digital education.
Universities have recently seen a surge in international students as their catchment areas widen. This surge has necessitated more remote testing as potential students are tested on their English language capabilities remotely before being admitted to UK universities. This challenge has so far been met by using a range of test centres operated by a variety of organisations on the universities’ behalf, but there is a huge potential for more flexible, convenient and robust remote invigilation services to be used here.
MOOCs are also likely to be a huge driver for computer-based testing solutions as Universities and their platform partners, such as Udacity and Coursera, seek to monetize the MOOC offer with certificates of completion, participation and, increasingly, learning achievement.
Recent fraudulent activities around qualifications have undermined the validity of MOOC courses and their certification. A robust method of assessment and invigilation is needed to give validity and security to the MOOC-testing process, and so to add value to these courses and certificates. Here, remote invigilation can provide this validity and authenticity, whilst also fitting in with the digital and remote requirements of MOOC users. Remote proctoring could enable MOOC providers to leverage the providence of well established brands, such as those of universities, to add further value to their courses.
From the above examples, we can see that there is already a desire, and need, for remote invigilation. However, to meet these needs we require a robust remote invigilation solution that is built for purpose.
Current Top Online Proctoring Services
There are already a number of companies who provide remote invigilation services worldwide, and as previously mentioned there are well-established online proctoring services in the US.
The table below gives information on the provisions and specifications of some notable vendors. Comprobo, Smowl and TestReach are UK/EU based, and ProctorU and Software Secure are US-based. The main difference between vendors is whether they use people to invigilate the tests remotely, or whether they use software to replace a human invigilator.
Comprobo
| Human Invigilation? |
| No, software replaces human invigilator. |
| Identity Verification |
| Facial Recognition Technology – a biometric picture is taken of the candidate’s face which is used to detect anomalies during the test. |
| Identity Verification 2 |
| Identity verification with e.g passport, driving license. A piece of I.D is photographed at the start of the test and compared to your biometric picture to ensure your identity. |
| Cheating Detection |
| Images taken at defined intervals with user’s webcam compared with biometric model to detect if candidate is the same person, if they are looking away, if there is another person present, or if they are not there. |
| Cheating Detection 2 |
| Tabbing detection – notification and optional failure if candidate opens a new tab or browser for more than a set period of time.
Optional target-clicking exercises to ensure that the candidate is paying attention. |
| Download or Browser Based? |
| Browser based |
| Integrations |
| Central to Comprobo it its powerful workflow-based capability. This includes question setting and content delivery. It is also able to invigilate activity on other websites through a monitored iFrame. Deep integrations with the major LMS and assessment platforms are in progress using LTI and QTI. |
| Vendor Location |
| UK |
Smowl
| Human Invigilation? |
| No, software replaces human invigilator. |
| Identity Verification |
| Images taken randomly with user’s webcam and compares with biometric model to detect if student is same person, if there is someone else helping, or if she is not in front of the screen. |
| Identity Verification 2 |
| Facial Recognition Technology – biometric testing begins when user logs in and is used to verify their identity throughout
Support team provides manual face recognition. |
| Cheating Detection |
| The facial recognition technology ensures that no one else is present during the test, and that the candidate is always looking at their screen, rather than using notes or other devices to cheat. |
| Smowl also uses an algorithm to detect if an unauthorised image or video is being used. |
| Cheating Detection 2 |
| Research in progress for iris-scanning and fingerprint identification systems. |
| Tabbing detection – notification and optional failure if candidate opens a new tab or browser for more than a set period of time. |
| Download or Browser Based? |
| Browser based |
| Integrations |
| Moodle, Blackboard, Totara and a general API to integrate the system into an LMS or web platform. |
| Vendor Location |
| Spain |
TestReach
| Human Invigilation? |
| Yes. Live online invigilation with a TestReach supervisor. N.B Automated invigilation in development. |
| Identity Verification |
| Identity verified through provision of I.D at start of test. |
| Identity Verification 2 |
| N/A |
| Cheating Detection |
| Monitored via webcam, audio, instant messaging and screen share.
Supervisor performs checks to ensure that candidate’s testing environment is satisfactory. |
| Cheating Detection 2 |
| When suspicious behaviour is detected an infringement is raised which is recorded and sent to the examining body.
Browser lockdown – candidate cannot open new browser or tab whilst test is in progress. |
| Download or Browser Based? |
| Download |
| Integrations |
| Integration with a variety of systems including HR, CRM and AMS. |
| Vendor Location |
| Ireland |
ProctorU
| Human Invigilation? |
| Yes. Connect with a live proctor who will be able to see candidate’s screen, as well as see and hear the candidate themselves. Candidate is required to show the proctor their environment – all 4 walls and the table they are using. Fully-automated, AI driven services are also available. |
| Identity Verification |
| ID must be shown to proctor to verify identity. |
| Identity Verification 2 |
| Personal questions, generated from public information database, asked to ensure identity. |
| Cheating Detection |
| Candidate can be monitored by a proctor throughout their tests, looking for eg – eye movement. |
| Cheating Detection 2 |
| AI technology can be used to detect suspicious behaviour. |
| Download or Browser Based? |
| Browser based |
| Integrations |
| Yes, widely integrated |
| Vendor Location |
| US |
Software Secure (RPNow)
| Human Invigilation? |
| Yes. Candidate’s test session is recorded and later reviewed by at least 2 certified proctors. |
| Identity Verification |
| Identity verified through provision of ID. |
| Identity Verification 2 |
| N/A |
| Cheating Detection |
| Restrictions can be enforced which prevent candidates from using certain programs during the test, e.g another browser, chat programs. |
| Cheating Detection 2 |
| Room scan conducted with webcam.
Report is provided to institution including any suspicious behaviour. |
| Download or Browser Based? |
| Browser based |
| Integrations |
| Can integrate with any LMS and test delivery system.
Certified by IMS. |
| Vendor Location |
| US |
Examity
| Human Invigilation? |
| Yes, uses human Proctor who watches examination. |
| Identity Verification |
| Identity verified through provision of ID. |
| Identity Verification 2 |
| N/A |
| Cheating Detection |
| Proctors raise ‘flags’ for suspicious behaviour, which instructors can review. |
| Cheating Detection 2 |
| Every test is audited after being proctored. |
| Download or Browser Based? |
| Browser based |
| Integrations |
| Not specified |
| Vendor Location |
| US |
Proctor360
| Human Invigilation? |
| Yes. Trained and Certified live proctors will be able to see the entire testing environment by using a 360-degree camera system, as well as with a front view of the candidate using their webcam. They can also monitor the audio. |
| Identity Verification |
| A government-issued ID like a driver’s licence or passport must be shown to the Proctor to verify and capture a picture of the candidate. |
| Identity Verification 2 |
| Based on the test client’s requirements, a second form of ID verification can be done, with options including signature and fingerprint capture. |
| Cheating Detection |
| The candidate is monitored by a proctor throughout the test, looking for suspicious eye movement, behaviours or sounds. |
| Cheating Detection 2 |
| Every test is recorded and can be further audited after being proctored. |
| Download or Browser Based? |
| Both download and browser based are possible |
| Integrations |
| A wide range of 3rd party software integrations are available via APIs. |
| Vendor Location |
| US |
The ideal remote invigilation solution
The aim of any remote invigilation solution is, as identified above, identity verification and invigilation.
Any remote invigilation solution which fulfils these criteria is useful, but it is the solution which does so most efficiently and robustly which will be most desirable.
Efficiency can broadly be classified by two further criteria: ease of use and cost-effectiveness. Ease of use will make a solution efficient because users and administrators will be able to use the system immediately without any trouble and associated troubleshooting, and an efficient system will be cost-effective as this is a desirable feature, and influential factor, for both the user and the administrator.
A solution is easy to use if it does not require any more knowledge or experience than the user already has. Typically, if a digital user has to download supplementary software or use new programs they will have to learn something new and will encounter issues, which slow down their use of the solution and often result in the need for additional support. A browser-based solution does not require supplementary software or the use of new programs and so a browser-based remote invigilation solution will be more efficient than one which requires a download.
A solution which is available to use at any time, rather than a solution which requires the user to adhere to certain time constraints, will also be easier to use. Automation is the name of the game here: if a remote invigilation solution uses real people to provide its services, then that service will have time constraints depending on when those people are operating. However, if invigilation can occur automatically, then a user can take their exam at any time, making an automatically-invigilated solution more efficient than human-invigilated one.
Automation will also have implications for cost-effectiveness. Once an automated solution is created, and if it works smoothly, then it will be much cheaper to operate than paying the wages of multiple human remote invigilators. An automated solution will therefore be more cost-effective than a human-invigilated one.
Privacy and data protection is major factor when you consider the personal data that is collected and stored as part of a technology-based invigilation solution. Comprobo has recognised this and organisations adopting Comprobo are able to specify where the data is stored. This can be a geographical location to take account of national regulation but it can also be at the organisations’ own data centre and therefor completely under their control and security policies. Smowl have addressed privacy issues by storing images on their own servers, which are deleted after a date specified by the client. These servers are based in Europe, although they can be moved based on the needs of the client.
An efficient remote invigilation solution will therefore involve browser-based operations rather than downloads, and will offer automated invigilation rather than human-invigilation.
Some recommendations
Learning Light has researched this market closely, we perceive that this market of remote invigilation is likely to grow strongly indeed and will unlock the next stimulus to the growth of e-learning
From our above table which examines popular remote invigilation solutions, we can see that Comprobo and Smowl are the most efficient truly remote invigilation solutions. As identified above, it is also important that a solution is robust, in terms of security and ensuring that remote invigilation is as effective as classic invigilation methods. Both Comprobo and Smowl provide robust solutions, using techniques such as biometric modelling and facial recognition technology, but Comprobo combines these techniques, and includes target-clicking exercises, in such a way that it is the most robust solution.
While Smowl offers a remote service, the actual invigilation process is really focused on identity management and facial recognition.
For a solution where human interventions are valued and support offered in terms of set up and real remote invigilation is offered, along with the option of an automated AI-driven solution, then our view is that ProctorU is currently the market leader.
However, making the most of the latest hardware and software technologies, Proctor360 is an exciting addition to the market and could soon lead the way by providing the most complete live monitoring of a candidate’s testing environment.
We’ll be watching closely to see who moves to capture this huge growth area within elearning and computer based assessment.
—
If you would like to find out more about the report, or how Learning Light can help you with in-depth market research and report-writing, please contact us.
