There have been some unfortunate developments in the last year. Currently, the Math Voice Recognition system I use requires 3 pieces of software (on Windows): Dragon Professional Individual, Scientific Notebook, and MathTalk. Unfortunately, the companies which make Scientific Notebook and MathTalk recently went out of business. So the software can no longer be purchased. Long-term (over the next 5 years or so), I (and others) will have to switch to some other system.
Given my reliance on such a system, I have already been researching a replacement. To be clear, no replacement technically exists. However, I looked into WYSIWYG math script program to replace Scientific Notebook (think Microsoft Word, but for math) and decided to transition over to Lyx (which is free). Lyx has the advantage that it is open source, and therefore will not go out of business in the future. But Lyx has no inherent voice recognition capacity. I have been in contact with the Lyx developer community, and have recently developed some scripting which will allow me to speak commands, which Dragon will then translate into LaTeX and insert that code into Lyx. I have generated scripts providing for the creation of most basic math functions, but it is early days, and I will most likely add much more going forward.
My plan is to get the functionality to a stable place, and then provide these scripts to the disability community. The scripts need to be imported into Dragon. Beyond that, there will be a learning curve. I plan to create a “dictionary” so that the user knows what commands can be spoken, and what the result will be. Currently, I just look at the list of scripts to remind myself.
Hey Jodin and to everyone having the same problem,
First I want to thank you for spreading the news. I am frustrated about this situation, too. I found out that there is an open-source software on GitHub which is called Mathifier. A few years ago a student developed a similiar software for math voice speech recognition.
https://github.com/batlounis/math-speech-recognition
It won’t be easy to install, but maybe you want to give it a try before creating a whole new software by yourself.
Kind regards,
Melanie
Hello Melanie, thanks so much for your response. I took a look at that github. I was encouraged. I’m going to have to think about how I might use that project, particularly when I am coding LaTeX. I’m still thinking that the project I have been working on is the direction I want to go, however. The reason is, I am looking for software that does for the user, something similar to what Microsoft Word does for regular text. When you use Microsoft Word, you don’t think about whatever the code is under the hood. And when one is trying to use their voice to do math, it is hard enough to think about the math, but it is more difficult when you don’t immediately see the math pop up as you say it. Rather, you see some code which needs to be compiled. I realize some systems might be able to turn it into visual math somewhat quickly (on a separate pane of the window). But I believe it should be as seamless as possible. So, speaking math into a program which represents the math without any noticeable code is of use. This is particularly true for young students with disabilities, who we shouldn’t ask to learn LaTeX before learning mathematics. That would present a barrier which would deter too many students.
I guess it would be nice to combine the projects. To be honest, even though I tried to figure this out on github, I’m not sure what engine it is using to do the voice recognition. I’m assuming it is using Google voice tech through a web connection? I’m not sure. Regardless, I wonder if Mathifier could generate the LaTeX, and then somehow be injected into the open source math presenter Lyx? That would be ideal. That way I wouldn’t have to reinvent the wheel when it comes to generating the dictionary of math expressions.
I found it, I see they are using Sphinx, a modular speech recognition tool from CMU.
When I first looked at the github, as you can read above had a misunderstanding of the project. I haven’t have this problem with github, as the projects are often not very well explained there. But I found a description at a different website. It looks like the project does present the math as math, and not as latex (even though latex is under the hood). I still haven’t found out how one implements it, or what it looks like. A video would be helpful. I’ll keep looking into it.
Unfortunately, I ran into the same problem I often run into with github. The authors did not make the installation instructions clear enough for a regular human to use. So I am not even sure how you would implement this package.
I don‘t have programming skills, so I asked a friend of mine to get to know how to install it. She is a professional software developer herself and is currently working on her promotion. She says herself that it‘s tricky :/
I currently use Lyx with Talon (also free). I had to download a seperate library given from Mathfly to the author of Talon and open for the community. Talon can be used with Dragon as well (though it has it’s own engine) from what I’ve heard and has an awesome community. It’s conveniently multi-platform and can even be integrated with eyetrackers.
https://github.com/mrob95/mathfly-talon
Hope this helps!
(PS. Everything works great on a mac, but I know PC users are figuring it out too)
This sounds like a great resource. I’m in the middle of the semester, so I can’t check it out now, but I will definitely check it out. When I do, I will report back! Thanks!
Thanks 🙂 I‘m going to try it, too.
Hi, any news on your project to dictate math? I’m looking for a simple solution for a young student who can’t type. I found many ideas online but none of them is easy to adopt: software is old or not existing anymore, compiling is not easy, there is no support for my language…
I hope you found something that suits you and can share your discovery!
Hello Ester, I will be giving a talk at MathFest in Indianapolis in a few weeks regarding my progress. It has been slow since I have been in a postdoc, and will now be transitioning to a permanent tenure-track position in New York this fall. My hope is to get some summer funding, or a semester release where I can spend some time finishing the project. The final result will be some scripts that you can import into Dragon Dictate That will allow you to use Lyx using your voice. I already have some of the more basic commands scripted, but it is insufficient to release at this point. It also does not yet come with a dictionary people can refer to determine what you must say to generate the mathematical script.