Novel Multi-Modal Tools to Enhance Disabled and Distance Learners’ Experience of Mathematics

— This paper discusses ICT-related needs, with particular reference to studying mathematically-based disciplines, of several types of users – notably people with physical disabilities, online (distance) learners and people working or studying “on the move”, relying on mobile devices. We note the inadequacy of existing interfaces for mathematical input and output for these groups and investigate several options for addressing the problems. We present two tools that employ novel human computer interaction methods which allow users to create and edit mathematical content in electronic documents and address some of the above issues. We apply one of these in a user study in a classroom environment, aiming to investigate whether this system can assist students to learn mathematical concepts via creating mathematical e-content. We show that there is some evidence that, by using this tool, some students can improve their general understanding of mathematical concepts, comparing “deep” natural language descriptions with representations based on “shallow” application-specific interactions.


I. INTRODUCTION
ver recent years, standards of literacy and education have been greatly improving in many emerging countries, contributing significantly to their economic development.However, many such countries have suffered from conflicts, leading to many young people losing limbs or the use of limbs.Some such disabilities are due to direct combat, but many are caused by exploding land mines, cluster bombs and similar munitions in conflict zones from Afghanistan and Angola to the former Zaïre.According to The Guardian [38], between 1999 and 2008 Landmine Monitor documented 73576 mine and unexploded ordinance (UXO) casualties Worldwide.Of these, around 55000 were maimed rather than killed, the majority being civilians and nearly a third of them children.In 2009 alone, over 2800 people were injured, in Manuscript updated on January 31, 2013.Dilaksha Attanayake & Eckhard Pfluegel are in the School of Computing & Information Systems, Kingston University, UK (e-mail: k0962524@kingston.ac.uk,E.Pfluegel@kingston.ac.uk) Gordon Hunter & James Denholm-Price are in the School of Mathematics, Kingston University, UK (e-mail: G.Hunter@kingston.ac.uk,J.Denholm-Price@kingston.ac.uk) addition to over 1000 deaths, due to such UXO [39].Furthermore, despite widespread immunisation and treatment campaigns around the globe, diseases such as polio continue to cause disabilities, cases due to such causes being more common in developing regions than in developed countries.(Polio, or formally poliomyelitis, in some central Asian and sub-Saharan African countries results in age-standardised Disability-Adjusted Life Year (DALY) rates in excess of 5 -in the case of one country, as high as 19whereas no developed country has a DALY rate due to polio greater than 1 [32].DALY is a measure of the average number of effective person-years lost due to disability caused by a particular disease, including contributions from "years of life lost" and "years lived with disability".)Disabilities due to the above, as well as other causes such as accidents, can seriously impair the educational and career opportunities of people who suffer from them.This may occur in an environment where a good education, leading to a professional career, can provide a route out of poverty.As discussed previously in [10], such opportunities in many disciplinesmost scientific and engineering fields, along with many in the commercial domain -are underpinned by achievement in mathematically-based subjects (at least at an elementary level).However, the teaching and learning of mathematics is considered very difficult by many people, especially those with a variety of special needs.Although assistive educational technology has provided help to various groups with such needs (see [47] for a review), the development and use of such technology in mathematical disciplines has been rather limited to date.This is in part due to the rather complicated two-dimensional layout of conventional mathematical text, plus the specialised terminology, notation and symbols used in mathematics.
In addition, creating & editing electronic mathematical content still remain difficult for both able and disabled students, and particularly for people studying "at a distance" (e.g. via Open University) and those relying on mobile computing devices such as smartphones and tablet PCs [21].The sophistication of Human Computer Interaction (HCI) methods in the domain of mathematics is still lagging behind those for other disciplines, meaning that the cognitive load required of people using conventional mathematical editors may have a considerable impact on the individual's learning of mathematical concepts.

O
This project aims to study how users create and edit mathematical formulae which are typed into our TalkMaths system (described in Section III) using natural language commands, rather than using conventional equation editing software (such as the Microsoft Word equation editor).In particular, our goal is to evaluate whether such an approach can aid participants' understanding of particular mathematical concepts, such as the "numerator" and "denominator" of fractions, to reinforce their understanding of these and related ideas.This is carried out through questionnaires, both to test participants' knowledge and understanding and to quantify their own perception of these, with respect to appropriate mathematical concepts.The current paper describes a small pilot study of this type within an introductory non-specialist mathematics course and measures the students' performance in both pre-and post-task tests.We intend to use the results of such studies in the development of new teaching and support materials aimed at improving students' understanding of key ideas.These activities will be rolled-into more courses in the future, which should give useful insights and data for additional studies.
The remainder of this paper is structured as follows.Firstly, we discuss the need for novel interfaces for creating and editing mathematical content in electronic documents and discuss several possible options for these.We then review the development of, and facilities provided by, our own natural language interfaces for mathematics, TalkMaths and SWIMS.We subsequently present details and results of an initial user trial to evaluate TalkMaths as a tool to help improve students' understanding of mathematical concepts.Finally, we put forward our conclusions and propose possible future developments.

II. NOVEL INTERFACES FOR THE TARGET USER GROUPS
Conventional human-computer interfaces based on input from a keyboard and mouse and output via a visual display are clearly far from ideal for the three target user groups identified above.This is particularly true for the task of creating and editing mathematical content in electronic documentsthe standard notation for representing mathematics and commonly-used mark-up languages (such as LaTeX and MathML) are not particularly accessible for these types of users.However, as noted by Österholm [36], the use of specialised symbols and notation gives mathematics great strengths, saving time and space relative to attempts to represent mathematical expressions in ordinary narrative language, which are likely to result in lengthy and potentially ambiguous descriptions.Österholm's study [36] concluded that reading and understanding mathematical text, in standard notation, required rather different skills, which took much time and effort to acquire, from the corresponding task on text written in ordinary language.Although conventional mathematical interfaces are constantly evolving and improving, the problem remains that such systems are not very user-friendly or helpful to naïve users.The learning curve is somewhat steep and lengthy for these systems and issues regarding ease and efficiency of their use are highly significant for our target groups of disabled, mobile deviceusing and on-line (distance) learners [21].The following subsections present some previous approaches to addressing these issues.

A. Solutions using Tactile Output
Braille has been a successful tactile medium for nearly 200 years, enabling the blind to read and write.Conventional Braille codes use a two-dimensional representation for each alphanumeric character, but these are arranged in a "linear" format, much like the normal orthography of English and many other languages, to represent words, phrases and sentences.This is not particularly well-suited to represent mathematics.However, novel extensions and modifications to Braille have allowed blind and other visually-impaired students a much wider access to mathematical resources (e.g.[37]).Nevertheless, all of these coding schemes need to be learned by both the students and teachers, which is a problem for most teachers since they (except specialist teachers of the blind) will probably only teach a rather small number of blind students over many years.Some previous authors have also investigated the use of other forms of tactile (haptic) interfaces (e.g.[33]) to replace visual output, or have implemented a hybrid interface combining both Braille and synthetic speech (see below), such as the LAMBDA (Linear Access to Mathematics for Braille Devices and Audio synthesis) projects [34,35].

B. Solutions using Optical Character Recognition (OCR)
For some groups, typing, but not writing, mathematics is a problem.These would include people with certain types of repetitive strain injuries, people using small mobile devices and, in some cases, on-line distance learners [21].It has been noted that a growing proportion of studying and academic exercises are being carried out using mobile devices such as smartphones and notebook or tablet computers, often "on the move" or in potentially noisy public places such as cafés [21].In such cases, input using a keyboard becomes difficult and use of spoken input using Automatic Speech Recognition (ASRsee sub section C below) is impractical.Thus, some users might prefer to create and edit mathematical expressions using a handwriting-based system.
For such people, an appropriate option might be to write the necessary mathematical expressions using a smart pen or stylus.The characters used could then be identified using an optical character recognition (OCR) system, and converted into (correctly) typeset mathematical text in electronic form.Some previous authors [26,27,28,29] have developed systems following this approach and indeed OCR has been included into the latest Samsung Galaxy Note II smartphone, with the aid of an innovative stylus called the S Pen [30].However, two remaining issues are; how to deal with the possibilities of misidentified symbols (potentially a big problem, since many people have poor "on-screen" handwriting) and mistakes by the user.Previous researchers have used syntactic [28] or statistical [29] approaches in attempts to resolve these issues.The latter approach is to some extent similar to the methodology (SLMs) we use in this paper (see section III E below).

C. Solutions using Spoken Input and/or Output
In recent times, speech has become a realistic alternative method for communicating with computers, and speech technology -especially synthetic speech output [48] and automatic speech recognition (ASR) [23,49] -is becoming quite sophisticated and reliable.Furthermore, humans' learning and teaching of mathematics [19] has not been simply a matter of silently reading and/or writing of the mathematical text, but also tightly associated with speech, gestures and other "side effects" for many years [18].There have been a variety of systems -primarily designed to assist blind and other visually impaired people -attempting to provide synthetic speech descriptions of mathematical text, including AsTeR (Audio Systems for Technical Readings [40], MathGenie [41], REMathEx [42], the commercial system MathPlayer™ [43], and AudioMath [44].The latter system is open-source but, unfortunately, only functions in Portuguese.The LAMBDA system [34,35], mentioned in section II A above, also gives the option of the automated reading-out of mathematical text by synthetic speech.In contrast, systems employing ASR (and further processing) to allow spoken dictation of mathematical expressions can benefit not only the visually impaired but also people with disabilities affecting their hands or arms, and people working or studying "at a distance" (e.g.on-line) or "on the move" using mobile devices [21].Previous approaches to allowing spoken input of mathematics include the research prototype systems of Bernareggi & Brigatti [45] (which only works in Italian) and Hanakovič and Nagy [46] (which is restricted to use with the Opera web browser), plus the commercial systems MathTalk™ [3] (which is only compatible with certain commercial editors) and Math Speak & Write [6] (which has a rather limited mathematical vocabulary).All of these systems have serious limitations, prompting us to develop our own system, TalkMaths.ASR is a key component of any mathematical interface which allows input via speech, and it can be carried out either on client side or on server side, or even partly on each side in a "distributed" approach.A more detailed discussion of issues relating to these options for speech recognition within the domain of mathematical text editors can be found in [1].

III. TALKMATHS & SWIMS
TalkMaths [4,7,8] is a web-based editing system for mathematical e-content that can be controlled using a variety of modalities (keyboard, mouse and speech).Our other prototype system, SWIMS, is a proof of concept for a web-based mathematical document editor developed to assist the user by intelligently predicting and/or correcting mathematical input by the use of statistical language models created from several sources of data [10,11].The remainder of this section will provide a technical overview of TalkMaths and SWIMS.We refer the user to our other papers [4,7,8,10,11] for further details of how they work.A demonstration video, showing TalkMaths in use, can be viewed at the webpage http://www.youtube.com/talkmaths.Figure 1 illustrates the system architecture of the TalkMaths prototype that has been used for this experiment.Two input modalities are available, namely spoken input (using ASR) and textual input from a keyboard & mouse.Each of these result in a text stream which describes the mathematical expression of current interest in relatively natural language (see section III B below).The text stream is then passed through the browser interface, encapsulated in a HTTP request, to the web server.The web application on that server then handles the so-called application logic (covering general security, session management and storage tasks) [1] and calls the parser, running on the Google Apps server [2], which ultimately processes the input text stream, transforming it into the desired format, such as MathML or XML.Again, web app requests and parser responses use HTTP.Upon receiving the parsed output from the Google Apps server, the web app sends the output (in the appropriate marked-up form) to the browser on the client for rendering in conventional mathematical notation.We have used a recursive-descent context-free grammar parser generator, called Yapps2 [5], to develop our TalkMaths parser.Due to the bi-modal nature of our system, and the consequential (small) differences between the correct spoken and typed descriptions of any given mathematical expression (see section III B below), a preprocessor has been used to convert the input from either spoken or typed form (as appropriate) into a canonical form that is suitable for analysis by our parser.

A. Multimodality
While SWIMS is only currently used with keyboard and mouse, TalkMaths accepts input from keyboard, mouse and via speech, which allows this new version of the application to reach a wider user base than its previous counterpart [8].

B. Natural Language Commands
Both SWIMS and TalkMaths employ an input language that is much closer to how people actually speak mathematics in a classroom environment, compared with specialised mark-up or formatting languages for mathematics, such as LaTeX or MathML.The rationale behind this is to make learning to command the system to be as easy and intuitive as possible for the user, keeping the "naturalness" of the task to a maximum.For example, Figure 2   Note that both forms of the TalkMaths command language are easy to read and easy for a person to speak/type.They should be much more accessible and easy to learn for novice users than either LaTeX or MathML.Also, note that the TalkMaths speech input language [8] requires use of the NATO alphabet [9] for the dictation of single characters, due to issues of potential confusion between conventional letter names by ASR systems (e.g."bee" (b), "cee" (c), "dee" (d)).An example of a simple mathematical expression is the equation for velocity under uniform acceleration v = u + at which in our spoken mathematical language would be read as: "victor equals uniform plus alpha tango".A more complex example is the formula for the solutions of a general quadratic equation: which would be spoken as "minus bravo plus or minus square root of bravo squared minus four alpha charlie all over begin two alpha end".Greek characters, such as α , β , etc. can be inserted using the prefix "greek" before the name of the character.For example, the trigonometric identity : sin( α + β ) = sin α cos β + cos α sin β would be read as "sine begin greek alpha plus greek beta end equals sine greek alpha cos greek beta plus cos greek alpha sine greek beta".An example of a more complicated expression which can be interpreted by TalkMaths is the van der Waals equation from thermal physics, rendered as shown in Figure 3 below.This would be dictated as "open bracket capital papa plus begin november to the power of two alpha end over begin capital victor to the power of two end close bracket open bracket capital victor minus november bravo close bracket equals november capital romeo capital tango".

C. Editing paradigms
TalkMaths allows users to edit mathematical expressions that they have input, as rendered on the computer screen, by issuing relatively intuitive commands which are close to how a human-to-human interaction would deal with the same tasks.For example, editing the numerator of a fraction can be invoked using the "edit numerator" command.Here, the application "understands" that the user requires the editing of only a part (in this case, the top part) of the fraction.Another example is where the "edit functions" command will invoke editing of all available functions within the current mathematical expression.We refer to these types of edit commands as "semantic editing" commands, as they refer to the meaning of the mathematical structure/components which the user sees on the screen.Other types of commands include "selective editing" and "exhaustive editing".The former is used to select specific sub-expression(s) within the full expression shown on the screen.For example, if a user wishes to change the sub-expression 2a in the quadratic formula (1), he/she can use the command "edit two alpha", which will make the denominator (2a) of the above quadratic formula editable by placing indexed bounding boxes around all possible selections of the sub-expression 2a -in this case, just the denominator.In contrast to this, exhaustive editing makes use of a displayed set of nested indexed boxes superimposed over the expression to allow the user to select the whole, or any part, of the expression for editing.Three such methods, highlighting all sub-expressions, all individual symbols and all operators, respectively, are illustrated in Figure 4.A more detailed explanation of these editing paradigms can be found in [4,20].

D. Natural Language Search-Driven Help Facility
TalkMaths has a Help facility that includes a natural language search option.A user can search through the help information using this tool, by entering search terms in natural language form.If the search phrase contains a word that is not in the vocabulary (V) of the Help content, we use the Damerau -Levenshtein algorithm [12,13] to calculate the Levenshtein distance between the entered word and each word in V.This distance is based on the minimum number of insertions, deletions, substitutions and transpositions of characters (1) required to transform one string into the other.The word within V with the shortest distance from the entered word is selected as the "best guess" for what the user intended to enter.
The Damerau -Levenshtein method was originally introduced to compare the similarity of text strings, and we found that this strategy works well when the user makes minor misspellings when typing using the keyboard and mouse.For each Help term, we assign a score based on its length, the length of the entered search term and how similar the terms are (based on the Levenshtein distance).Our mechanism assists the user to find appropriate commands using their existing mathematical knowledge.For example, typing, "How to create a fraction" command will present all the TalkMaths commands associated with fractions and rank them according to how relevant they are (see Figure 5).

E. Intelligent Prediction & Correction
A user is bound to make some mistakes during any typing or dictating process.This is likely to be an even bigger problem when entering mathematics than for ordinary text.In contrast to what is required for a conventional text editor, a mathematical text editor has to encode the complex spatial layout and specialised symbols of mathematical expressions.However, the user may need to type or dictate the content in a linear manner.For example, the linear statement/command "fraction a over b end fraction", or just "a over b", should be displayed on the screen in the conventional non-linear mathematical layout: Such differences between the spatial layout and format of the input and output make the editing and correction much harder for mathematical text than for conventional "ordinary" text.
From our earlier work, [7,8], we found that mathematical text can be reasonably predictable within a sufficiently large dataset.We have used this observation to develop a prediction feature into SWIMS [10,11] so that a user can get intelligent assistance from the system in order to help them type mathematical expressions quickly and efficiently.This prediction mechanism, which is in some ways analogous to predictive text in ordinary word processing systems and on SMS text editors on mobile 'phones, is based on N-gram Statistical Language Models (SLMs) [16,17,23], which have been key components of most successful ASR systems for many years.N-gram models are based on statistics of sequences of N consecutive words in previously encountered example ("training") text of the appropriate naturein our case, descriptions of mathematical expressions using the nearnatural command language described in section III B above.We have also included a semi-automatic correction system, based on the Damerau -Levenshtein method [12,13], which prompts the user with suggested corrections when an out of vocabulary word is observed in the input.These features are analogues of predictive text and auto correction facilities in ordinary text editors.In SWIMS, these facilities offer likely alternatives, in transcribed "spoken mathematics" form and/or displayed in conventional mathematical notation, to what the user actually entered (see Figures 6 & 7 for examples).Fig. 6.Predictive Mathematics Interface in use.In the top-ranked suggestion, the SLM predicts that "charlie" will be followed by "end".Fig. 7. Alternative/Corrective Mathematics Interface in use.In the top-ranked suggestion, the OOV word "pluk" is replaced with "plus".

F. Research Challenges
The general case of natural language parsing is believed to be an unsolved problem.However, whilst our systems, TalkMaths and SWIMS, require the parsing of "close-tonatural" language, namely spoken mathematics and typed mathematical language, in both these situations we are dealing with a much more controlled case than for general natural language.In these cases under study here, we have a restricted vocabulary (of the order of 100 words) and a fairly prescriptive syntax which, with the exception of a few "pathological" examples, can be well-described by a contextfree grammar [4].Nevertheless, incomplete or ill-formed mathematical input can be highly ambiguous and is much harder to parse and then render compared to well-formed, unambiguous mathematical expressions.We have used a topto-bottom LL(1) parser algorithm [24] and pre-and postparser error recovery strategies in our prototype version of TalkMaths for the work discussed in this paper.Currently, we are looking into novel ways of parsing mathematics by using only the precedence of operators [22] with non-deterministic GLR (generalized-LR) parsing techniques [14,15].However, in this paper we do not discuss the technical advances made by using these methods.

IV. EXPERIMENT
In this section, we present the design, implementation and results of an experiment on using TalkMaths with real students in a real classroom environment.This was to assess the TalkMaths application in terms of its usability and impact on learning of mathematical concepts compared with use of a conventional mathematical editor.A user evaluation had previously been carried-out on the original, desktop-based version of TalkMaths [4].However, this focused on the system's ease of use and how fast and accurate users were in performing various mathematical editing tasks.It was found that the majority of participants, who did not have any disability, took longer and produced more errors using TalkMaths than when using a conventional keyboard & mouse based editor.Nevertheless, the only participant who did have a major disability (Duchenne Muscular Dystrophy) performed better, both in term of speed and accuracy, when using TalkMaths, and out-performed many of the non-disabled group when using this modality [4].This illustrated the potential benefits of TalkMaths to one of the user groups for which it was primarily designed.The present study investigates how the new version of TalkMaths influences the users' understanding of mathematical concepts related to the prescribed editing tasks, instead of speed and accuracy in performing the editing.The results of a preliminary evaluation of the new version have already been published elsewhere [31], but scrutiny of these revealed some weaknesses in the design of the original experimentnotably ambiguities in some parts of the questionnaires.These limitations were addressed and the revised materials were tested using a new group of students.The details and results of this refined, follow-up study are presented below.

A. Design of Learning Activities
We developed a set of classroom mathematical materials and learning tasks for undergraduate Life-Science students who were taking a basic mathematics module at Kingston University.The tasks to be carried out by the participants, and the questions on their mathematical knowledge, were designed to be appropriate to their typical level of mathematical expertise.The volunteer student participants were allocated to two groups randomly.Both groups carried out the same tasks, but using two different tools.The first group (A) used a conventional editor (Microsoft Word Equation Editor) while the other group (B) used our research prototype system, TalkMaths.Note that all subjects had previously used MS Equation Editor but none had used the TalkMaths system before.Each participant was asked to complete three questionnaires.The first questionnaire was about the participant's own perception of their mathematical competence at the start of the experiment.The second was a diagnostic test related to the tasks they were about to carry out and the final questionnaire, given at the end of the exercise, was similar to the second, in order to assess improvements to the participants' understanding, but also included questions concerning their experience of using whichever system they were allocated.From our pilot experiments [31], it was observed that some participants omitting to answer some of the questionsparticularly in the post-task questionnaire -led to results of dubious reliability.Hence, in the follow-up study we instructed the subjects that all questions on both pre-and post-task questionnaires were mandatory.Furthermore, after detailed scrutiny of the original versions, the revised questions were re-designed and re-worded to be as unambiguous as possible.Details of the tasks and the questionnaires are given in the appendices.

B. Undertaking of Learning Activities
The regular teaching staff and demonstrators for the module supervised the participants carrying out the tasks of the experiment, without actually instructing them.The students were required to learn by themselves, with the only resources available being the instructions given in the worksheet and the Help facility of whichever tool they were assigned to.After completing the first two questionnaires, participants were required to undertake three tasks (Tasks 1, 2 & 3see the Appendices) creating and editing mathematical expressions involving fractions, functions and square roots respectively.Each task required them to create a specified equation and then carry out a minor modification to this using the editor they were assigned.As noted previously, all participants had used Microsoft Equation Editor in earlier practical sessions but no prior training on TalkMaths was given.(Thus, we expected "better" performance and possibly higher levels of satisfaction amongst the group using MS Equation Editor.)All participants were encouraged to use the Help facility of the editor to resolve any questions they might have while completing the tasks.

C. Evaluation of Learning Activities
Table I presents the results on the post task feedback from the participants on the ease of use of the tool they were using to complete the tasks.The group using the MS Equation Editor seemed to find it easy to use, more-so than the group using TalkMaths.When these qualitative evaluations were each put onto a 5 point Likert scale, the differences in perceived ease of use of the system by the two groups proved just to be statistically significant (p ≈ 0.047 in a one-tailed ttest).However, this was to be expected due to the participants' previous experience with MS Equation Editor, in contrast to none of them having used TalkMaths before.Table II presents the results on the post task feedback from the participants regarding any improvements they perceived to their understanding of the mathematical concepts involved in the task after using the appropriate tool they were allocated.
Most users of TalkMaths (61%) seemed to believe it had improved their understanding, while only 35% of users of MS Equation Editor reported any improvement.In fact, 14% of users of MS Equation Editor thought it had impeded their understanding, whereas no TalkMaths users held this opinion.However, when these qualitative evaluations were converted to a 5 point Likert scale, the difference between the two groups' perceptions proved not quite to be statistically significant (p ≈ 0.065 for a one-tailed t-test).
We also evaluated the students' performance on knowledge of relevant mathematical terminology, both before and after the tasks, to investigate whether the exercise, possibly including use of the Help facility of whichever tool they were allocated, had improved this knowledge.We gave a score of +1 whenever the participant correctly gave the prescribed answer to one of these "mathematical knowledge" questions, or a score of +0.5 for an answer we considered to be a relevant "near miss".If the answer given was incorrect or the student did not respond, a score of 0 was given for that question.We observed the change to each student's score between the pre-and post-task questionnaires.Although, in both groups, most individuals gave the same response to any given question in both pre-and post-task questionnaires, indicating no change to their knowledge, one participant in group A (MS Word Equation Editor) actually did worse the second time, indicating a decline in understanding !In the previous, pilot study [31], such observations could possibly have been due to laziness, where the subject did not complete all the questions in the post-task questionnaire.However, this was not the case in the current study.Although the overall average score for the group did increase slightly after performing the tasks, this improvement was very small and was not statistically significant (p ≈ 0.18 in a one-tailed ttest).In contrast, no participants using TalkMaths did worse on the second questionnaire, five students improved their scores and overall the group average score increased by a noteworthy amount, which proved statistically significant (p ≈ 0.011 in a two-tailed t-test, or p ≈ 0.006 in a one-tailed test).The difference between the two groups in mean improvement after performing the tasks also proved to be weakly significant (p ≈ 0.048 in a one-tailed t-test), where the TalkMaths group showed greater improvement.

D. Discussion
We found that use of the TalkMaths system did make a modest, but statistically significant, improvement to the average of the students' scores on their knowledge and understanding of the mathematical concepts relevant to the given tasks.They also perceived this to be the case in their own qualitative self-evaluations of how they had performed.Although both the mean knowledge score and students' selfperception of their understanding improved by a small amount for the MS Equation Editor group, neither of these were significant statistically.This suggests that TalkMaths has better potential as a tool to aid people's mathematical skills than does MS Equation Editor.At first sight, the results of this study on the ease of use of the two systems might appear disappointing, with students on the whole finding the MS Equation Editor easier to use than the TalkMaths editor.However, it must be born in mind that all the students had used MS Equation Editor before, whereas none of them had any previous experience or training in the use of TalkMaths.In the future, we would hope to perform a more controlled experiment, either using participants who had not previously used either editor, or providing some initial training in the use of TalkMaths, in order to reduce any bias in favour of one system or the other.Furthermore, previous researchers have investigated students' preferred methods and styles of learning.Some such studies have suggested considerable variations in the ways different individuals learn.For example, Fleming [25] has identified four rather different learning styles adopted by students, categorising these as Visual, Auditory, Reading/Writing and Kinesthetic within his so-called VARK model.It is also possible that differences in individuals' preferred learning styles, and other similar factors, might affect whether an editor which is conventionally text-based, GUI-based, or which employs a more novel modality (such as TalkMaths) would be best suited to a particular user.As we have already noted, different modalities may also be appropriate for users with various disabilities [4,8], people relying on mobile devices, and online (distance) learners [21].Some users may prefer to create and edit mathematical expressions using a handwriting-based system with optical character recognition (OCR).Some researchers have already investigated this [26,27,28,29], and indeed OCR has been included into some recent mobile devices, including the latest Samsung Galaxy Note II smartphone [30].

V. CONCLUSIONS
This paper has discussed the ICT-needs, with reference to studying mathematically-based disciplines, of certain groupspeople with disabilities, online (distance) learners and people relying on mobile deviceswhich represent a notable section of the population in the emerging regions of the World.We have discussed various ways in which some of these needs can be addressed and presented two innovative tools which make some progress towards achieving these goals.These have been employed in a user evaluation study, with real students as participants, comparing our own mathematical text editor, TalkMaths with a conventional, commonly-used commercial system (MS Word Equation Editor) for ease of use and participant opinion on whether it had improved their mathematical knowledge and understanding.We also performed quantitative assessment to investigate whether there was any actual evidence of such an improvement.The results from this initial evaluation suggest that there is still significant room for improvement for TalkMaths in these respects.However, although fewer users found our system, TalkMaths, easy to use than the more conventional MS Equation Editor, TalkMaths did appear to have a greater beneficial effect on the participants' knowledge and understanding of mathematical concepts.This was the case both from the users' own perception of their mathematical understanding, and from the results of pre-and post-task testing of their mathematical knowledge.It should also be noted that all the users in the study had used the conventional editing system (MS Equation Editor) previously but none had used TalkMaths before.This fact could contribute to biasing the users' opinions in favour of the conventional system.Furthermore, none of the participants in this study were really from the target groupsnamely the disabled, distance learners and people relying on mobile devicesfor which TalkMaths was designed to help.We would expect more positive results if participants were drawn from such groups of users.In the light of these observations, the results of this study are quite encouraging, bearing in mind that TalkMaths was a system of which the users had no previous experience.

VI. FUTURE WORK
In addition to the work described in this paper, we are investigating novel parsing strategies for spoken mathematical expressions in the hope of obtaining more powerful, reliable and robust analysis of natural language descriptions of mathematics.Once these expressions have been rendered into conventional mathematical notation, further innovative methodologies for editing them, both in terms of the paradigms the user can employ to carry out the necessary modifications, and assistive tools such as predictive or corrective text, will be implemented and evaluated.
In the future, we are also aiming to carry out a large scale usability study on our TalkMaths system, with participants from the target user groups: we would like to present our editing paradigms to experienced speech-interface users, disabled users, distance learners and mobile device users and evaluate how user-friendly and useful the system is for them.Eventually, we would like to integrate TalkMaths features with other systems, including editors, mathematical and educational toolssuch as computer algebra systems -to enhance the experience of creating, using and modifying mathematical content easily and intuitively, whilst also assisting people's learning and understanding of mathematical concepts.

Fig. 4 .
Fig. 4. Different editing paradigms for editing mathematics by speech, each applied to one of equations of uniformly accelerated motion.
shows encodings of the same mathematical expression, namely , in LaTeX, MathML, TalkMaths spoken input form and TalkMaths & SWIMS keyboard input command language.

TABLE I HOW
EASY IS THE EDITOR TO USE?