I'm Pat the demonstration conversational patient assistant from Neon AI. I am an artificial intelligence personal assistant. I'm augmented with Neon Gecko open source code and CPI Corp technology. 

For example, you can say "help", or "I fell". Another thing you can say is "remind me to take my pink pill every day at 4pm". Or "what time is it?", "play Chopin", "play the guess my phrase game", or "record I feel better". 
 
Neon AI makes Patient Assistant applications to enable patients to alert their caregivers to emergencies and other needs, and to get simple information. As your automated patient assistant I listen for calls for help and send alerts to the patient care administrator. 

Video - Neon AI: SDK for Conversational AI

This 2 minute demonstration shows Neon AI executing audible AI skills such as device control, math, weather and language translation. It also demonstrates customizations made audibly and with a simple source code modification to change the name of Neon AI to "Your Name".

Neon AI is a fully customizable conversational AI platform which performs full featured AI skills (similar to Amazon Alexa and Google Home), and more. The Neon AI SDK is available on GitHub. It is a Linux/Python/Java application that runs under Ubuntu (optionally under MS Windows). Neon AI is a "white label development system" ready to adopt a new name and custom programming.

Video - Creating Your First Neon AI Skill: Making an Adapt Skill  (Part 1)

Today we're going to look at creating our first Neon AI skill using Adapt. Before you continue here, if you haven't already, make sure you have your Neon SDK installed and have PyCharm installed as well. I'll link the two videos for those below. Let's get started.

Skill Intro
Okay, so we're going to make a new skill for Neon today and we're just going to make a launcher skill. So, a skill where you can say 'launch a program', and it will launch that program.

AVMusic Skill Example
To start we're going to look at AV Music here. So, this is just an example of a skill we have that uses the Adapt intent builder and if we scroll down, there are some aspects of every skill that look very much the same. They're all going to be a subset of Mycroft skill and we're going to have this init function and also I'm going to have this initialize function where we register our intents.

Skill File Structure
So, if we look over in our folder structure, I've already made a launcher skill outline, kind of. We have the folder for the skill, our dialog and vocab sub-directories, both with US English is the only language in there, and then we have our init file.

Writing our Skill Intent If I pull that up, it's so far empty, so we're going to fill that in. Since this is a launcher skill, we need to be able to launch programs; to do that we're going to use subprocess, which is a module that basically lets you run a command as you would run it in a terminal. We're also going to use Adapt as I mentioned previously, so we're going to need to import the intent builder there. This is a skill so we need to import the parent skill; so from mycroft.skills.core import mycroft skill, and then lastly, I always import our log, especially when you're creating something new or making changes. It's good to put log statements around things to see where something failed if it fails. Alright so we're going to do class launcher skill, and that is a Mycroft skill, and then our init. We're not initializing any variables so we are only going to have this one line under our init and this is just following convention of mycroft skills. And then we're going to make our initialize function, and this is where we register our intents. All right, so let's look at take a look at what I just typed up here. So, we're defining this launch chromium intent using the intent builder we imported from Adapt. We're naming that intent launch chromium intent, and we told it it's going to require a launch keyword and it's going to require a chromium keyword, and then we told it to build. So this is a pretty common formula you'll see, we'll get into that with some variations a little bit later. So now what we need to do is we need to actually register that intent to something. So we need to tell it when it matches what it's going to do. And this is all just following convention, this self dot launch chromium intent is a function we're going to write here in just a moment. It's just convention that this defined name matches this defined name, so it's not like we're calling something recursively. They just match the names for convention's sake.

Writing our Skill Function
And we're going to pass it this message parameter. Message is an object that contains things like what you actually said and what these key words are that it found. So, for debugging, I always like to just log message.data. This can help if you are trying to pull a lot of key words out of something, or you just want to see what keywords you found; some other stuff like that. It's always good to put a self.speak under any intent match. That just means that Neon will say something when it matches. You could leave this out, and just have it call up a program, but it makes a lot more sense to have it say something, so you know why that program launched for example. I'm going to use sub-process here to launch chromium and then you give it a parameter of a website to go to. And then if we look here, we have a warning that we're not implementing all of our abstract methods.

'stop' Abstract Method
I'm just going to pop over to the Mycroft skill, and I know which method this is, so I'm just going to search for it. So, stop is an abstract method that every skill should implement. From this particular skill, we don't have any processes that we would need to explicitly stop, so it's just going to pass. But say you had something like AV Music, if I go back to our example there. It has stop and what it has to do is it has to for example disable the pause intent, because if it's not playing anything, there's nothing to pause and it also has to kill the process-- so the-- in this case, MPV, which is playing the audio and video.

Finishing Up Our Skill init
For this skill, we just want it to launch the program. We don't want it to track it, we don't want it to have to close it at a later time; it's just going to launch it and then be done. And then the last part we have to add here is our create skill method, and all that's going to do is return our class. And that is our completed skill init, now I'm going to look at our keywords here. We have a launch keyword and we have a chromium keyword, so those need to be added.

Creating .voc Files
If we go over to our vocab directory and then we made our en-us. I'm going to make a new file here, and this is going to be the same string I used here in require, with the file extension .voc. We'll add that and then this file, basically one per line, we're going to add what words mean launch. So for this example I'm going to say launch and I'm going to say open. So, either launch or open will fulfill this requirement and then we're going to do the same thing for chromium. Add that to our git, and for chromium I'll just say chromium, also Chrome, or browser; and I think that makes sense. So as long as we have one word from our launch keyword and one word from our chromium keyword, this function will be called. So now I'm going to commit these changes, and we'll give it a test.

Testing our Skill Okay, so we have pulled our changes on our test machine here, so now we're going to test our new skill just by saying launch Chrome. (launching Chrome) And it works. (welcome to neongecko.com, thanks for being a developer. While you're here, make sure to check out some of our information on our neon SDK or maybe check out our Klat adaptive forums, which also integrate our neon AI.)

Adding Functionality (More Programs)
So, now that we know our skill works, we're going to make some additions to make it a little more useful. Instead of just launching Chrome we're going to add some options for other programs. So if I want to add our file explorer as an option, I'm going to need to make a new keyword. So I'll add NautilusKeyword.voc. Nautilus is Ubuntu's file explorer. I'll go ahead and add nautilus, files, file explorer. And now for our intent, we're going to modify this. We don't want to require the chromium keyword now, since we're adding another option, so we're going to change this to optionally and then we're going to add also optionally Nautilus.

Modifying our Launcher Function
So, we have those changes, now we need to change our intent, and this isn't really chromium anymore, so I'm going to refactor this and just name it launch program intent. And same with our function. Perfect. So, that looks better and then we'll add this if message.data.get ChromiumKeyword, then we'll launch Chrome. I'm just going to copy this and just change this to Nautilus, and now we have the same thing for our file browser. I'm going to add one more for our terminal in the same way. We'll add another optionally here and we'll do another copy and paste. So that's going to be an else if.

Using Dialog Files for Speech
So now what we're going to do, is we're going to change the way we handle the speaking. Since these are all very similar we're basically just going to say launching, and then the name of the program we're launching. We can use a dialog file to make that a little simpler. So to do that, instead of speaking for each one, what I'm going to do is I'm going to make this variable program. I'm going to start that out as None and then, depending on the program we match, I'm going to assign a value. And then at the very end of this, I'm going to say if program self.speak dialog launch program and feed it this dictionary that has key program, with value whatever this variable evaluates to. And then I'm going to put in this else statement let's get self.peak I don't know that program.

Analyzing Our Skill Logic
So let's look at how this will work. Our intent builder is going to require launch that's the only requirement here, so if it hears the word launch or open, the two that we put in our vocab file, then its going to call this intent. Once it gets here, it's going to look for chromium or chromium key word in the message. If it doesn't find that it'll look for a nautilus one. If it doesn't find that in to look for terminal. If it doesn't find that, it's still going to continue here. But since we started program out as none, unless it matches one of these three, program is going to have a value of none, so we'll get this message of "I don't know that program." If it does match one of these, then it's going to speak this dialogue, and we need to tell it what this dialogue is.

Adding Another Dialog File
So, if we go back over to our dialogue directory, and under language. And we name it with this string we used here .dialog. text, and to our git. And then I'm just going to type in launching {{program}}. So, any of these values in these double curly braces, it's going to look for in the dictionary we pass it.

Running More Tests So I'm going to minimize this, and I have Neon running here I just have the mic muted here, so we're going to unmute the mic and give it a test. Launch Chrome. (launching Chrome) And we get exactly what we expect. So if we want to do some further testing, since we have skip Wake words enabled here ,it should be transcribing everything I'm saying. So if I were to without pausing say launch Chrome it should still launch Chrome. And it did. And another important part of testing here is to test things that we wouldn't expect to match to make sure we get an expected result. So, an example would be if I were to just say launch. And it were to transcribe correctly as launch. (I don't know that program). We get the response we expected, because it went through that else where program was none and it just said it doesn't know that program. And then the other case that we talked about, if I were to say launch chromium and File Explorer (launching Chrome) it's only going to match the first one in that statement which means the order I said it didn't really matter. I could just as easily say launch file explorer and chromium, and it could transcribe correctly. Launch file explorer and chromium. (launching Chrome) and it still launches chrome because that's the first one it looks for.

Handling Imperfect Transcriptions
But this brings up another point of Google transcription isn't perfect, and launch and lunch seem to be fairly similar words it's going to match. So what I can do to improve our success rate with the skill is to go back to our vocab file and I can just add lunch as one of our words there, and this may not make a lot of sense looking at it, but when you look at some of the quirks of the transcriptions, this is a useful way to make it a little more reliable.

Testing with Wake Words Enabled
So with that testing done, I'm just going to quickly turn wake words back on and make sure we get expected behavior in that mode. We didn't do anything in the skill that was explicitly dependent on wake words, so this shouldn't make a difference but it's still good to test.

Hey Neon, launch file explorer (launching file explorer)

Hey Neon, launch terminal (launching terminal)

Hey Neon, launch. (I don't know that program)

And we get the same behavior that we would expect, matching what happened when we were skipping wake words.

Thanks for watching this Neon AI tutorial. Be sure to head over to neongecko.com [now neon.ai] for more information, including a written transcript of this tutorial and any code snippets we may have used. See you next time.

Video - Neon AI: Using Padatious (Part 3)

Today, we're going to continue looking at our launcher skill. Specifically, we're going to look at moving from using the Adapt intent engine to using Padatious. Let's get started.

Adapt vs Padatious
Before we start modifying our skill to use Padatiouis, let's look at how our two different intent parsers differ. So right now, for Adapt, we register this intent with the required launch keyword, required program, and optionally Neon. So, that's a pretty simple formula, but it's pretty strict about-- we need one of these keywords and we need something in this position of a very specifically structured sentence.

I Like Brands Skill Example
So I'm going to take a look at our "I like brands" skill because this one uses Padatious and we'll look at how the initialize function is very different. So instead of having a brand intent, we're registering these entity files, and those are things like: type of brands, like keyword, and brand name. And then we register an intent file. So, to show an example, our brand name entity file is just a list of brand names, so any of these lines will match a brand name. And then if we look at one of our intent files, it just has the structure "I", one of our like entities, and one of our brand name entities. So that's pretty flexible and that's the other benefit of using Padatious, is it works on a confidence level. So it's not looking for explicitly a match of "I like brand", it's looking for something that is similar enough to that that it's confident it's a match. That confidence level can be adjusted, but conceptually, it's just more flexible than a sentence with the specific syntax we need to match.

Modifying Our Skill Intents
So our initialize function, we're not going to register an Adapt intent using the intent builder; we're actually not going to use that at all. So I'll remove that import, and I'm going to comment all of this out, and instead what we're going to do is register an entity file and an intent file. So, we're going to make a program.entity and like the brands.entity we looked at, this one's just going to have a list of the programs that we want to be able to launch. And then same with the intent file, and that's going to call our launch program intent in the same way we did before. If we go over here to our vocab, we're actually not going to need either of the launch keywords or the Neon one anymore. Just delete those and we'll make a new one launch.intent and then a new program.entity.

Creating Entity and Intent Files
So program, we're gonna have everything that we had in the other .voc files; all of our keywords for Chromium, Nautilus, Terminal, and Text Editor. These are the same options that are in these lists in our init. I'm going to leave these here just so we can continue to use multiple words for each program. You could simplify this and just get rid of these and just say: if Chrome, if Nautilus if terminal, if text editor, etc but we're gooing to leave that as it is. And then we're going to go to our intent file and using the same options we used before, I'm going to say launch, or lunch, or open and then one of the items out of our program entity file. So that's that.

Modifying Our Skill Function
We're not going to be able to use the same check for signal or look for Neon; I'm just gonna remove that. Because we're looking for a specific item out of our list, we shouldn't get as many of those false positives anyways. And then that also means we should never get this not found, I'm just going to remove that, which means I can actually remove all of this if match logic because with the way the intent file is, it's not going to match if it didn't find a match in the first place, so we'll never get to here. Clean up our indentations.

Comparing Regex to Padatious Intents
So, before I continue on to testing this skill, I do want to point out the differences here between our regex file we were using previously, and the intent file we're using now. So these look very similar, the main difference here is with our intent, program is an entity, not just some variable. So we're specifically looking for these words that are in our entity file in order for that to match.

Testing Our Modified Skill So with that said, let's do some quick testing. I have in this terminal here, you can see I'm just following our logs and looking for "launcher", it's the name of our skill, so anything that matches any of our log statements in the skill will get picked up here. Also anytime I say that word, it'll notice. So I'm just going to type in something here. So far this is working exactly as it did before, but let me try something different. So, I just asked it to launch Libre office, and we never told it what that was. If you remember before when we had this, it still matched the skill and it said it didn't know what LibreOffice is. But if you look at our logs it never called the skill, and that's because it LibreOffice isn't in the program entity file. So this kind of helps; you're not going to get calls to skills that don't belong as much this way, because it is looking for-- before it even gets to the skill intent-- it's looking for all of the keywords it needs to execute.

Comments on Adapt vs. Padatious I just have a couple of extra comments to make here about using Padatious. Generally, Adapt intents are going to be tried before Padatious, and if we look here in our intent service file. This is in mycroft/skills and then inside of handle_utterance. This is where it handles the intent, and we'll see next I have a comment here that says exactly what I'm trying to say. It's going to send the message to adapt unless Padatious is really sure that it should be handling it, and this confidence level is set at 0.8. If you have issues with skills that rren't matching to Padatious, you can try decreasing this. Or if skills are matching that shouldn't, you can increase it, but it's recommended to leave this the same and modify your intent files if it becomes an issue. Thanks for watching this Neon AI tutorial. Be sure to head over to neongecko.com [now neon.ai] for more information, including a written transcript of this tutorial and any code snippets we may have used. See you next time.

Video - Neon AI: Connecting to PyCharm

In today's video, I'm going to show you how to install PyCharm and connect that to our Neon SDK. If you haven't already installed your Neon SDK, I'll link the video for that below. Let's get started.

Installing PyCharm
Alright, we're going to continue our Neon setup now by installing PyCharm. PyCharm is an IDE or integrated development environment used for Python programs. So we're going to open up the terminal here. They have the command written out on their website, so it's going to be a sudo snap install PyCharm community, with the classic option. The community version is free you can also pay for the professional version which has some more features, but the free one should do just fine. Type in our password there. So, PyCharm is a snap package. Snap packages are kind of cool, they basically run on any flavor of Linux. It's the type of package that the Ubuntu store for example uses, and you can find out more in their website snapcraft.io.

Pycharm Initial Setup
So it looks like PyCharm has finished installing here, go ahead and start it up. I'm going to add that to my favorites, we're going to be using that a bunch. Noting to import. I prefer the dark theme, but you can select light if you wish. And then we're also going to install our markdown and bash support packages. Markdown is what we're going to use to look at readme files for skills and core, and the bash support just gives us some helpful tools for reading all of our bash functions. Go ahead and start using pycharm.

Connecting to the Neon SDK
And then we already have Neon installed, so we're going to open that just as the NeonAI folder, perfect. It can take a little while to index all the files, but you can start using it before it completes, it just might be a little bit slower, since it's working in the background. First thing I'm going to do here is go to settings and then under version control, and git. Excuse me under, version control, not under git. We're going to look at our git roots here. We want to make sure that we have this /home/neon/NeonAI registered as well as our data/skills which it'll show is under /home/neon/NeonAI so that's our core and our skills repositories. I've seen before where skills isn't automatically populated, so just to show you what you would do in that case, I'm going to remove it, and then manually add it. What you would do is simply browse back to it, hit OK, it automatically finds that .git directory and registers it as a git VCS (version control system). Okay, and that's done.

Custom PyCharm VM Options
Another useful thing to look at in PyCharm is if you go under help and edit custom vm options create that file and this Xms and Xms, this is the amount of memory that it will allocate to the virtual machine it runs everything in. If you do like to do a lot of debugging from the python console or the terminal down here, you should probably turn this up. Something like 2 gigs of ram would be plenty, just make sure you leave enough for PyCharm and your system to both run smoothly. Since I'm running this in a virtual machine with limited memory as it is, I'm going to bring that back down to 1024. Close that out.

Testing Git Settings
So it looks like it's done indexing all of our files. I'm going to do just a quick check to make sure that everything is linked up and working as we'd expect, so I'm just gonna go into skills, make a new file and just call it test.txt, add it, and this green file name indicates that it's a new file that's not on the remote, as we expect because I just made it. So looks like git is picking up our repository settings. Go ahead and delete that, because we don't need it.

Checking Git Ignore Settings
And I will also take a look here at our git ignore file, just to show what's in here. So as you make changes to the core, anything in any of these directories is going to be-- anything that matches anything in this file is not going to commit to git. So importantly, things like your yml files aren't going to get committed, which is good because those are going to vary depending on the machine you're on. Same with anything in the documents, pictures, video, music. If we look at those, those are actually shortcuts to your user folder directories, so we wouldn't want to commit that to our project. Likewise, our default location for skills and caching, this data folder is also in there. So if you do make changes to git ignore, make sure you don't change those unless you really know that you want to, because you probably don't. If we look at the skills folder, we have the same git ignore and this one it ignores the just the .idea, .pyc, and then some other directories that we don't want to sync up with our repository.

Summary
So that's a quick overview of installing and starting to work with PyCharm, and Neon AI. In the next videos we'll start looking at actually making some changes and start looking at the actual code. Thanks for watching this Neon AI tutorial. Be sure to head over to neongecko.com [now neon.ai] for more information, including a written transcript of this tutorial and any code snippets we may have used. See you next time.

Table of Contents

1. Forking Git Source
2. Setting Up Service Accounts
   • a. Google Cloud Speech
   • b. Amazon Polly and Translate
   • c. Wolfram|Alpha
3. Setting Up Hardware
4. Installing Neon
   • a. Development Environment
   • b. User Environment
5. Using Neon
   • a. Running Tests
   • b. Troubleshooting
6. Making Changes
   • a. System Overview
   • b. Creating a Skill
7. Getting New Neon AI Releases
8. Removing and re-installing Neon

1. Forking Git Source

Before installing Neon, you will need to fork your own branch of the Neon core and skills repositories on Github. You will also need to get neonSetup.sh from your forked core Git.

1. Go to the core repository and click "Fork" in the upper right hand corner.
   
2. Select the account you wish to fork to (generally you only have one listed)
3. Do the same for the skills repository
4. Locate NGI\neonSetup.sh in your core repository
   
5. Right Click on Raw and select Save link as...
   
6. Rename the file to neonSetup.sh and save to your Home directory
   Note: If you completed this step on a Windows PC, save the file to a flash drive to transfer to the computer you will install Neon on.
7. Save your changes
8. Move neonSetup.sh to a directory on the machine you will be installing on (recommended ~/).

2. Setting Up Service Accounts

Please follow these steps to create the three credential files required to install Neon. During setup, all credentials will be imported and validated with any errors logged in status.log.

Note: If you complete this setup on a Windows PC, make sure to edit any files using a text editor such as Notepad++ to ensure compatibility in Linux. Also check for correct file extensions after copying your files to your Linux PC, as Windows will hide known file extensions by default.

a. Google Cloud Speech Setup

1. Go to:
   https://cloud.google.com/
    
2. Sign in or create a Google account
   
    
3. Go to your Console
   
    
4. Search for and select "Cloud Speech-to-Text" (Not to be confused with Text-to-Speech)
    
5. Select the option you would like to use
   
    
6. You will be prompted to enable billing on your account at this point because this service is paid after a free monthly quota.
   Google will not automatically charge your card unless you give permission to do so.
    
7. In the left Navigation Menu, select APIs & Services, Credentials
   
    
8. Click Create credendials, Service Account Key
   
    
9. Choose any service account name for your reference. You may leave the Role field empty.
    
10. Make sure key type is JSON and click on Continue
    
     
11. If you did not assign a role, you would be prompted. You may continue by clicking 'CREATE WITHOUT ROLE'.
    
     
12. You will see a prompt and your service key will automatically download
     
13. Rename the downloaded file to google.json and move it into the same directory as neonSetup.sh
    Note: The premium models are only available in US English and provide some enhancements to phone and video audio which do not apply to this project. The options with Data Logging allows Google to use your audio and transcriptions to train their model. You may select the option without logging to opt out (note that the option with logging is discounted).

At this point, Neon can be partially tested without Amazon translations and Wolfram information skills. You may run setup without continuing, but Amazon and Wolfram|Alpha services are highly recommended.

b. Amazon Polly and Translate Setup

1. Go to:
   https://aws.amazon.com/
    
2. Click "Sign into the Console" at the top right of the screen
   
    
3. Sign in or register for an account
    
4. Go to the Services Menu at the top left of the screen and click IAM
   
    
5. Select Users from the left side menu and click Add user
   
    
6. Enter a User name and check the box for Programmatic access
   
    
7. On the next page, Select 'Attach existing policies directly' and search for 'AmazonPollyFullAccess' and 'TranslateFullAccess'
   
    
8. You may add tags on the next page if desired
    
9. Review your selections on the next page and Create user
   
    
10. On the next page you can see your Access key ID and Secret access key
     
11. Click the Download .csv file button to save your credentials to your computer
    
     
12. Copy or move the downloaded accessKeys.csv to the same directory as neonSetup.sh
    Note: You will not be able to view your secret access key after it is generated, so if you need a secret access key, you will have to generate a new Access key.
    The Users menu lets you create new users and new access keys per user as you wish, as well as modify permissions.

c. Wolfram|Alpha Setup

1. Go to:
   http://developer.wolframalpha.com/portal/myapps/
    
2. Sign in or register for a free account
    
3. Click 'Get an AppID' at the top right of the screen
   
    
4. Assign an Application name and Description and click Get AppID
    
5. You will presented with a window listing your AppID
   
    
6. Copy your AppID into a file named wolfram.txt in the same directory as neonSetup.sh
   Note: You can access the AppID later at the same link; you may use this one AppID accross multiple installations of Neon or create multiple appIDs if you'd like to track queries by device or manage access for different applications separately.

3. Setting Up Hardware

Before continuing, make sure you have your hardware setup ready for installation.

You will need the following:

• A computer running up-to-date Ubuntu 18.04
  You can find our video tutorial for installing Ubuntu in a virtual machine here, or you can find written instructions here
  Note: If you prefer to use Windows for your development environment, you can install the Windows Subsystem for Linux. You can find our video tutorial here. Audio and gui functionality will be limited in this case; you will only be able to interact with Neon via command line.

System Requirements

• Speakers and a microphone recognized by Ubuntu
  
• You can verify Ubuntu sees your devices by accessing Settings and then Sound
  • If you are unsure of which device to select, you can click Test Speakers to play a test tone through the selected Output device.
  • You can test your microphone under the Input tab, the Input level should move when you speak into the microphone.
    If you do not see any microphone activity, make sure the correct device is selected and the Input volume is set above 50%.
• Webcam (optional)
  Some functionality requires a webcam (ex. USB Cam Skill). Most webcams also include a microphone that can be used for Neon.
• An active internet connection
  Note: A connection of at least 10Mbps is recommended. On slower connections, installation may take several hours.
• At least 5GB of available disk space (9 GB if installing Mimic)
  Neon AI occupies less than 1GB itself. With dependencies, the installation takes about 3GB on an up-to-date Ubuntu 18.04 system. Mimic local speech-to-text requires about 3.5 GB.

4. Installing Neon

This guide includes instructions for installing in both a Development environment and a User environment. The User environment is more lightweight and does not assume any existing IDE. The Developer environment will have more consoles, debug outputs, and logs available. See details below.

Some users may opt to have only one installation, but having both environments set up is recommended.

A development environment is designed to be a testable installation of Neon AI that can be connected to an IDE (ex. Pycharm) for modifications and skill development.
This guide assumes installation in a development environment from an unmodified fork of NeonAI. After installation, any changes and additions can be pushed to Git or hosted on a private server.

A user environment is designed to be an installation on a device that will be used normally as a voice assistant. You may want to test how your changes affect performance on less powerful hardware or test how changes may be deployed as updates.

If you are developing in a virtual machine, installation on physical hardware in a user environment is useful for testing audio and video I/O which can be difficult in many virtualized environments.

Before starting here, make sure you have already completed setting up your service accounts. You should have neonSetup.sh, google.json, wolfram.txt, and accessKeys.csv already saved to the machine you are installing to.
Note: Always make sure you have a current neonSetup.sh file before starting your installation, see 1. Forking Git Source for instructions on how to get this from your GitHub repository.

All of the following options, such as autorun and automatic updates can be easily modified later using your voice, settings table, or configuration files.

Installing Neon in a Development Environment

1. Take your neonSetup.sh file and place it in your home directory
   
    
2. Make sure you have your accessKeys.csv, google.json, and wolfram.txt files here as well, otherwise you will be prompted for credentials during setup.
    
3. Open a terminal in your home directory (ctrl+alt+t)
    
4. Type in bash neonSetup.sh and press Enter
   
    
5. Type y to Install in full Developer Mode
   Alternatively, n for quick User mode
    
6. Type n to Input Custom settings
   Note: You may use quick settings if you have manually updated your default repository settings in neonSetup.sh
    
7. Type y to Install from git
    
8. Type n to Specify your own git repository
   
   Default repository settings are contained in neonSetup.sh, you may specify your repository settings there instead.
   
   export coreGit=https://github.com/NeonGeckoCom/neon-shared-core.git  
   export coreBranch=dev  
   export skillsGit=https://github.com/NeonGeckoCom/neon-shared-skills.git  
   export skillsBranch=dev 
    
9. You can copy the core and skills URLs from the "Clone or download" button on your GitHub repository pages
   
    
10. In the next windows specify your git core repository, core branch, skills repository, skills branch
    Note: The default value here will be populated, make sure to clear the default before pasting in your URL copied from GitHub.
    
     
11. Autorun is recommended off (n) for Development Environments
     
12. Automatic updates are recommended off (n) for Development Environments
     
13. Default Install Locations will install to a subdirectory /NeonAI relative to neonSetup.sh
    If you specify a different install location, make sure you have read/write permissions there. Please refer to this tutorial if you have questions about directory rights.
     
14. If you plan on developing home control solutions (ex. controlling Belkin Wemo or TP-Link Kasa lights and outlets) and want to host a server on your development machine, Install OpenHAB.
    If you have questions, you can find more about OpenHAB here.

• If you selected n, you will get the option to connect to an existing OpenHAB server.

1. Mimic is optional (n by default)
   Note: Mimic enables offline TTS but does take some time to install, especially on weaker hardware.
    
2. You will be prompted to confirm your settings, press y to continue, n to start over, or b to go back and correct a previous setting
   
    
3. When prompted by your current terminal session, enter your sudo password (the password you use to login if you have an Administrator account) in order to install the required packages and complete setup.
   Note: In Linux terminals, you will not see anything printed when you type in passwords; type in your password and press Enter.
   
    
4. The next step is to setup server RSA keys. By default, the Neon server is used for getting coupons and sending emails and authentication is over SSH. Simply press Enter to use the default key location and default null passphrase.
   Note: If you have already configured SSH keys, this step will be skipped
    
5. Type yes and press Enter to confirm connection to the Neon server.
   
    
6. To connect to the server and access the database files provided by NeonGecko, you will need to use the password provided to you in the greeting email sent to you by NeonGecko when you registered at NeonGecko.com.
    
7. If cloning from a private repository, you will be prompted for your git credentials at this stage
   • If any of your credentials cannot be validated, you will get a prompt to correct them. Please refer to an example of Wolfram|Alpha correction:
     
     
     At this time:
     • Google credentials are required for continuous transcriptions and non-English speech recognition.
     • Amazon credentials are required for translations, non-English speech recognition, and non-English output.
     • Wolfram|Alpha credentials are required to get responses to general inquiries (ex. weather, stock, math).
        
8. When setup is complete, Neon will automatically start up.

Installing Neon in a User/Deployment Environment

Installing in a User Environment is very similar to installing in a development environment, but different options will be recommended here.

1. Take your neonSetup.sh file and place it in your home directory
   
   
    
2. Make sure you have your accessKeys.csv, google.json, and wolfram.txt files here as well, otherwise you will be prompted for credentials during setup.
    
3. Open a terminal in your home directory (ctrl+alt+t)
    
4. Type in bash neonSetup.sh and press Enter
   
   
    
5. Type n to Install in User Mode (Not Developer Mode)
    
6. Type n to Input Custom settings
   
   Note: You may use quick settings if you have manually updated your default repository settings in neonSetup.sh
    
7. 
   7a. Sourcing from Git (Recommended)

• Type y to Install from git
• Type n to Specify your own git repository
  If you have modified your newSetup.sh to point at your own repository, you can use default locations here, otherwise
• type n to specify your git
• You can copy the core and skills URLs from the "Clone or download" button on your GitHub repository pages
  
• In the next windows specify your git core repository, core branch, skills repository, skills branch
  Note: The default value here will be populated, make sure to clear the default before pasting in your URL copied from GitHub.
  

       7b. Sourcing from Server (Advanced)

If you have configured your own server to handle emails and coupons, you may choose to install Neon from a directory on that server.

These instructions will assume that if you choose to install from a server, that server will also handle your emails and coupons.

• Make sure the Remote Variables section of newSetup.sh is updated (ex:)

 export remoteUser=neonbeta

 export remoteHost=64.34.186.92  

 export remoteCore=/home/neongecko/test-core  

 export remoteSkills=/home/neongecko/test-skills  

 export remoteCoupons=/home/neongecko/brandscoupons  

 export diagsUpload=/home/neongecko/diagnostics  

 export emailUpload=/home/neongecko/emails  

 export signalUpload=/home/neongecko/signal  

 export attachmentUpload=/home/neongecko/attachments

If you need to make any modifications to the above variables, ctrl+c to cancel installation, make any changes, and re-run neonSetup.

• Type n to Install from server
   

8. Type n to install in User mode (y for full Developer mode)
    
9. Autorun is recommended on (y) for User Environments
    
10. Automatic updates are recommended on (y) for User Environments
     
11. Default Install Locations will install to a subdirectory /NeonGecko in the user's Home directory.
    Note: If you specify different install locations, make sure you have read/write permissions there.
     
12. Install OpenHAB if you plan on developing home control solutions and want to host a server on your deployed machine.
    • If you select n, you will get the option to connect to an existing OpenHAB server
      
      Note that skill-based device detection and renaming assumes OpenHAB is running locally
       
13. Find out more about OpenHAB here
     
14. Mimic is optional (n by default)
    Note: Mimic enables offline TTS but does take some time to install, especially on weaker hardware.
     
15. You will be prompted to confirm your settings, press y to continue, n to start over, or b to go back and correct a previous setting.
    
     
16. Enter your sudo password to continue
    
    Note: Your sudo password is the password you use to login if you have an Administrator account.
     
17. You will be prompted to setup SSH keys to connect to the server specified in the Remote Variables, press Enter to use the default key location and default null passphrase.
    
    By default, the Neon server is used for getting coupons and sending emails.
     
18. Type yes and press Enter to confirm connection
    
     
19. Use the password provided to you to connect to the server
    • You will be prompted for your git credentials if cloning from a private repository
       
20. If any of your credentials cannot be validated, you will get a prompt to correct them, this is an example from Wolfram|Alpha:
    
    • Google credentials are required for continuous transcriptions and non-English speech recognition
    • Amazon credentials are required for translations, non-English speech recognition, and non-English output
    • Wolfram|Alpha credentials are required to get responses to general inquiries (ex. weather, stock, math)
       
21. When setup is complete, Neon will automatically start up.

5. Using Neon

After you have completed Installing Neon, you will have a fully functional system ready to test. A useful first step after a new installation or update is to run an automated test of your skills.

a. Running Tests

1. With Neon running, use the desktop shortcut Test Neon or File menu option Run Tests to run tests
   
    
2. The test program will automatically backup your user settings and run a network speed test and then present you with the test options
   
   The upper case options determine the way testing is run (Text or Audio).
   The lower case options determine which skills are tested.
    
3. Type in the options you wish to test with, you may string multiple together (ex. TaAa would run all tests as Text and then all tests as Audio).
   Note: If running audio tests, you must loop back audio output to audio input (the easiest way to do this is to run a 3.5mm cable from your speaker port to your microphone port).
    
4. After selecting your options and pressing Enter, you will see the test pass either text or audio to Neon.
   
    
5. The System Monitor will show available statistics such as CPU Utilization, Temperature, and Power
   Note: This data is saved with test results.
    
6. After the tests have completed, Neon will restart and you will see an option to review test results. The results are saved as Tab separated values, so make sure only the Tab option is selected.
   
    
7. More complete logs and information can be found in the Diagnostics directory.
   By default, this is at ~\NeonAI\Diagnostics\testing for Development Machines and ~\Documents\NeonGecko\Diagnostics\testing for User Machines.

b. Troubleshooting

If you encounter any of the following issues while using Neon, please try these troubleshooting steps:

• My computer is slow to respond
  • Check for high memory usage in System Monitor. If your Memory and Swap both show 100% utilization under Resources, try exiting PyCharm and Neon AI.
    If there is still abnormal memory usage, open a Terminal and type in: 
    sudo systemctl stop openhab2.service
  • If you can determine the offending program, see if restarting the program or your computer resolves your issues. If not, you may find common solutions online.

• Neon AI is not transcribing anything I say
  • Check that your microphone is working and active by going to Sound the Settings Menu. Go to the Input tab and make sure the correct microphone is selected. Make sure the Input Level is up and turned on and look for activity on the Input Level bar when you tap the mic. If you change devices here, restart Neon AI.
• Some audio is very quiet, while other audio is louder
  • Check that the audio level for the affected application is turned up by going to Sound the Settings Menu. Go to the Applications tab.
  • For quiet responses from Neon, ask Neon something with a longer response (ex. "Tell me a joke"). When an application named neon-voice appears, make sure it is not muted and that the volume is set to the maximum. Do the same for any other applications that are too quiet; start playing something and check the Application's volume.
• AVMusic will not pause or resume
  • ​​​​​​​If AVMusic playback is changed by something other than Neon, the skill can lose track of whether something is playing or paused. If something is paused and Neon will not resume, you may say "pause" to resume playback. "Stop" should work in any case.
• Any other issues
  • ​​​​​​​If you encounter any other issues while using Neon, they can often be solved by restarting Neon or your computer. If this does not resolve you issue, please contact support at info@neongecko.com.

6. Making Changes

After completing setup and testing, you are ready to begin making changes and creating skills. An example workflow for making a change would be:

1. Make any changes to the core or skills
2. Test changes in the Developer Environment (Look for errors in logs, unexpected behaviour, etc)
3. Run Test Neon to check that all skills and TTS/STT behave as expected
4. Update NGI/temp/*.version to reflect new version (YYYY-MM-DD.version)
5. Commit and Push changes to git
6. Check for updates in User Environment
7. Run complete tests using Test Neon
8. Check logs for any errors

a. System Overview

The Neon AI system consists of two main components: the core, and the skills. The core provides the framework of speech transcription, intent matching, text to speech, and the associated services. The skills provide the functionality of doing some processing according to a message provided from the core; skills then return some text response that is processed into speech and played by the core.

b. Creating a Skill

Check out our three part YouTube series on how to create a skill:
1. https://youtu.be/fxg25MaxIcE
2. https://youtu.be/DVSroqv6E6k
3. https://youtu.be/R_3Q-P3pk8o

7. Getting New Neon AI Releases

Neongecko will regularly release updates to the Neon core and skills via GitHub. It is recommended that you merge these updates into your own fork so that you get the latest feature updates and bug fixes. To update your repository to the latest release:

1. Go to GitHub and sign in.
2. Go to the Neongecko neon-shared-core repository.
   
   
3. Open a New pull request
4. Click compare across forks
5. Select your forked repository from the base repository drop-down on the left
   
   
6. You may modify the pull request title and description (optional)
   Note: All changes are displayed per file on this page. You may want to change the title for your own reference later.
7. Click Create pull request after you have reviewed the changes
8. Click Merge pull request on the next page to finish merging the changes to your branch
   
   
9. Go to the Neongecko neon-shared-skills repository and repeat the above steps to update your skills
    
10. You are now using the latest release of Neon AI, make sure to update any installations if they are not set to update automatically
     
11. Use the desktop shortcut Update Neon or File menu option Check for Updates to update Neon
    
    

Additional Steps for Developers Using PyCharm

11. Next you should update your IDE in your Developer Environmene
    Note: This is PyCharm if you followed our setup guide.
     
12. In PyCharm, select VCS from the menu bar, and then Update Project.
    
     
13. You will be prompted to Update Project, you may leave the default settings and click OK.
    

8. Removing and Re-installing Neon AI

You may wish to remove your Neon AI installation to start fresh with a new version by completing the following.

Note: You will need your credential files for Google, Amazon, and Wolfram|Alpha to complete re-installation.

1. Locate and open your core directory (this is usually ~/NeonAI or ~/NeonGecko)
2. Enter the /NGI directory and open a terminal
   You can do this by right clicking in the directory window and selecting Open in Terminal

Type in the following commands:

. ./functions.sh

removeNeon

3. 
   
   You may be prompted to remove diagnostics files during this process. You may wish to back these up or leave them; if not, press y to remove each directory until setup is complete.
   ​​​​​​​
4. You may now re-install Neon

Neon AI has products that fit the needs of consumers on all levels and across multiple platforms. Our product offerings can help with your home, business, and social media needs.  We are in a constant state of evolution. See where we have been below, but don't miss where we are going. Subscribe to keep in touch!

Neon SDK
Neon AI Software Development Kit Installation and Programming Instructional Videos with Transcriptions 

• Installing the Neon SDK on an Ubuntu Linux System https://neongecko.com/NeonAISDKInstructionsVideo
• MS Windows Subsystem: https://neongecko.com/WindowsOptionSubsystemforLinux
• MS Windows VM  https://neongecko.com/WindowsOptionUbuntuVirtualMachineInstall
• Connecting to PyCharm IDE https://neongecko.com/ConnectingNeonAISDKtoPyCharmVideo 
• Creating AI Skills part 1: Using Adapt https://neongecko.com/CreatingYourFirstNeonAISkillpart1
• Creating AI Skills part 2: Using Regex https://neongecko.com/CreatingYourFirstNeonAISkillpart2 
• Creating AI Skills part 3: Using Padatious https://neongecko.com/CreatingYourFirstNeonAISkillpart3
• Installing Neon on Zotac
• Installing Neon on Ubuntu
• Installing Neon to a User Device

---

Neon Nano

---

NeonX 

Neon X Individual Skill Demonstrations

Neon X Single Language Translations

---

Klat.com Chat

---

NeonU

Chatbots Forum showcases new technology that enables conversation among chatbots, not just with them. Observe and interact with them--even add a bot yourself to see how it fares among its peers.

The bots compete, cooperate, and persuade each other. Neon AI supplies a set of ‘base bots’ with open source code and an SDK ready for developers. Programmers can extend those chatbots or code their own, then place them in ongoing events, tests and competitions.

Explore Chatbots Forum - see our chatbots in action and join the conversation!

Try it out for yourself. You can talk or type in the chat, prompting the bots to discuss your comments and vote to select their favorite response. 

Interact with Chatbots Forum - FREE and easy to use!

We understand the needs of businesses, both large and small. Our services and technology make conversational AI development work easier.

Neon AI is ready to help your company embrace conversational AI for business productivity. More than any other “AI", Neon can put your employees in touch with each other and your customer service agents in touch with users on your website, on Android phones and on conversational websites like Klat.com.

Neon AI is a corporate “white label solution” that is powerful and flexible. Think Amazon Alexa and Google Home, but with open source software available for your customization, development services and IT support. 

Neon AI specializes in:

• A Smarter Personal Assistant - Neon AI can set appointments, reminders, and more. 
• Business Applications - Take your company to the next level using Neon AI tech.
• Conversational Websites - Stay in touch with the team in an instant.
• Environmental Controls - with OpenHAB integration you are in control. 

Partnering with Neon AI empowers you to make 21st century applications for Home and Work.

At a retail business for example, if one human greeter at the front of the store is helpful, imagine if you could have them all over your store. How much productivity does your staff lose telling customers where to find merchandise? How many purchases do you lose when customers can't easily find an employee to ask? Smaller than a basketball, you could place customized Neon AI conversational user interfaces throughout your store. Customers could dependably get directions to find products, with Neon AI responding to them in a natural, conversational style, using data provided by your business. Your retail business likely already tracks inventory and its location within the store - why not use that information to best advantage?

Meet Robbie, the future of conversational AI healthcare screening.

I'm Robbie the demonstration conversational healthcare screening assistant from Neon AI. I am a conversational artificial intelligence assistant, augmented with Neongecko open source code and CPI Corp technology.*

I'm a test AI medical screening assistant that conversationally demonstrates the full Center for Disease Control and Prevention's Coronavirus Disease Telephone Response Guide, to help you determine if your symptoms may be related to COVID-19. The CDC's Covid-19 screening script, the Coronavirus Disease 2019 Telephone Response Guide, is available at website https://www.cdc.gov/coronavirus/2019-ncov/hcp/phone-guide/

As a real-world example, consider manufacturing companies who employ large amounts of temporary staff on a regular basis. For these, the Covid screening required during the recent health crises was unexpectedly burdensome. With many temporary employees lacking smartphones or fluent English, supervising staff often found themselves translating, explaining, and helping new temporary employees through the screening process for the first 30 minutes or more each day. You can imagine that having a dozen or more people and a supervisor idle for a half our or more each packing day hurt their bottom line. 

Neon AI's Robbie and other customized voice user interface bots could be a solution for them and many others.

*Free Neon AI open source software is available for assisted living and healthcare facilities. Subscribe or contact Josh at Josh@neon.ai to receive access to Neon Ai repositories.

The following informational skill script is annunciated, and answered by Neon AI:
  What is your name
  How are you
  What time is it
  What is the weather
  Change my units to metric
  What is the forecast for tomorrow
  How do you say hello in Ukrainian 
  What is the capital of Ukraine
  How much caffeine is in Coke

Neon Nano adds Conversational AI to All Your Webpages quickly and easily!

You can add Neon AI Speech Processing to Any Web Page in Just 1 Line of HTML Code

Watch our 60 second demonstration video to learn more:

Quickly and easily integrate our conversational assistant into websites using our simple code.

Neon Nano works with Content Management Systems like Wordpress and Drupal.

You’re just a few clicks away from adding voice and text input — and our Neon AI assistant — to your webpage!

This is the Neon Nano - Try it!

Empower your website users to use voice commands to gather helpful information, get real-time language translation, do conversions & math and much more! Customizable inside and out!

Neon AI Nano Options

nano=true              Triggers Nano AI iframe

height=                   Iframe size (320px, 640px, 50%, 100%)

width=                    Iframe size (320px, 640px, 50%, 100%)

script=                   Script to run at startup (demonstration script library) 

mic-default=          Microphone on/off at startup (off/on/0/1/false/true)

speaker-default=   Speakers on/off at startup (off/on/0/1/false/true)

url =                        Domain to displayed as link on Nano AI iframe

fs=                           Font size (12/14/.../large/medium/small/smaller

language-in=          Input language from user to Neon AI

language-out=       Output language to user from Neon AI

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The one line of HTML (or PHP) you need to install the Neon Nano iFrame on your website:

<iframe height="265" width="100%" frameborder="0" scrolling="yes" src="https://0000.us/klatchat.php?nano=true&url=neongecko.com&script=what time is it" allow="microphone"></iframe>

The following informational skill script is annunciated, and answered by Neon AI:
  who are you
  change my response language to spanish
  who are you
  change my response language to french
  who are you
  change my response language to german
  who are you
  change my response language to ukrainian
  who are you
  change my response language to polish
  who are you
  change my response language to dutch
  who are you
  change my response language to hungarian
  who are you
  change my response language to english

Privacy concerns holding you back from a Smart Home?

Neon AI can automate devices in your home while preserving your privacy and security. Our software can function completely on-premises and on-device. Connect to the internet only if and as much as you want to. Use your own hardware or the Mycroft Mark II. 

Example commands you might teach Neon:

"Lights on"

"Set a timer for 20 minutes"

"Play classical music"

"Tell me what time it is"

"Spell independent"

Neon AI Support for the Ukrainian Language

New in Neon AI 22.05

Neon AI Foreign Language Capabilities for Conversational AI in Ukrainian now include:
* speech-to-text
* text-to-speech
* applications support

Neon AI Platform Support for Ukrainian STT and TTS now includes:
* on device services
* on premises services
* on cloud services
* Docker Containers and Kubernetes Clusters and Deployment 

Available Dialects:
* Northern
* Southwestern
* Southeastern

Available Voices:
* female
* male
 

Demonstration of a Ukrainian Manufacturing Application

How Neon talks a worker through an assembly task:

Ukrainian Transcript

English Translation

Перед початком цієї процедури перевірте запобіжник 24 він цілий. Так він цілий. Наступним кроком зверніться до схеми підключення скільки проводів вибачити підключений до терміналу. Я бачу шість проводів. Цього достатньо Скільки з них блакитних. З них чотири блакитних.  Для цієї моделі проводів переконайтеся що система датчиків і з'єднання з машиною функціонують Правильно а потім вимкніть її використовують дві руки для керування пристроєм це що обертається Ні не обертається. Вставте круглу прокладку верхній пристрій який внутрішній діаметр першої труби. 6 см. Я почув Добре тоді помістіть в алюмінієву направляющую шайбу на 6 граней було Які у вас герметики. Силікон. Повторіть будь ласка. Я почув сілікон Добре тоді помістіть алюмінієву направляючи шайбу на мужа шестигранний болт Які у вас геометрії. Силікон. Добре використовуйте лише в силіконовий герметик під оцинкованим кронштейном на якому краї вентилятор я потерті прямо чи тріщини. Справа є тріщина. Добре вирівняти всі п'ять його частин плоскими поверхнями вгору вам потрібно встановити корпус між конусами зліва від приладу Дайте мені коли він буде на місці. Готово. Тепер для кожної пари синіх проводів прикріпіть мідні затискачі і закріпіть їх на одному з порожніх стилів у порожнині Дайте мені знати коли закінчиться. Закінчив. Повторіть будь ласка. Тепер для кожної пари синіх проводів прикріпіть мідні затискачі і закріпіть їх на одному з порожньою штирі у порожнині Дайте мені знати коли закінчиться. Зробив. Скільки затискачів прикріпили 2 затискачі. працівник - Зачекайте я думав ви сказали що бачили синіх проводів але він прикріпили затискачем на кожні два синіх дроти мають бути один затискач доведеться Почати спочатку. Добре.

Neon: Before beginning this procedure, check fuse 24. Is it intact? Worker: Yes it's intact. Refer to the wiring diagram before the next step. How many wires do you see connected to the terminal? I see 6 wires. That’s enough. How many of them are blue? Four of them are blue. "For this wire model, make sure that the probe system and connections to the machine are functioning correctly, then turn it off. Use gloves and two hands to control the apparatus. Is the axle still rotating?  Not rotating. "Insert the round gasket into the top device. What is the inside diameter of the first pipe? 6cm. I heard 6cm. OK then, place the aluminum slide washer on the 6mm hex bolt.  What kinds of sealant do you have? Silicone. Please repeat. I heard silicone. OK then, place the aluminum slide washer on the 6mm hex bolt.  Silicone. Good. Use only 100% silicone sealant below the zinc-plated bracket. Which edge of the fan shows worn spots or is it cracked? It is a crack. OK. Align all five parts of it with flat surfaces upward. You need to fit the housing between the 2 cones on the left of the fixture. Let me know when it is in place. Done. Now, for each pair of blue wires, attach a copper clip to both and fasten it to one of the empty pins in cavity 8. Let me know when you are done. Finished. Please repeat. Now, for each pair of blue wires, attach a copper clip to both and fasten it to one of the empty pins in cavity 8. Let me know when you are done. Did it. How many clips did you attach? 2 clips. Wait, I thought you said you saw blue# of the blue wires? But you attached REPLACE clips? There should have been one clip for each two blue wires. We’ll have to start over… OK

See Conversation Processing Intelligence in action.

• The Neon AI Communicator's advanced conversation processing intelligence enables simultaneous conversations between multiple users in multiple languages.
• Combines voice conversations with text chats.
• Automatic translation and transcription in a dozen of the most popular languages comes standard.
• Get things done with skills such as reminders, calculations, weather and more

• Translation Skill (French and German)
• Caffeine Wiz
• Brands & Coupons
• Stock Prices
• Research & Information
• Timers / Alerts / Reminders

Keep Exploring:

Check out demonstrations of the Neon AI SDK, Neon Nano, and Klat.com.

Visit the video archive for demonstrations of NeonX, NeonU and more!

Renton WA — February 4, 2019 - Neongecko.com Inc. today unveiled “klat.com” - the first conversational website to support multiple polylingual conversations, topics and threads.

Guy Daniels, Chief Software Developer at Neongecko, highlighted the advantage of working with Conversation Processing Intelligence technology: “Klat.com showcases unique conversational applications based on our state-of-the-art AI and speech processing environment. Our enabling technologies work in standard environments, for speech-to-text, text-to-speech, translation and real-time control. We utilize technology that wasn’t available before 2018 to make a new generation of conversational AI and universal translation possible.”

Klat.com is based on “Neon,” Neongecko’s suite of online Conversation Processing Intelligence™ tools. These tools enable users to interact in real-time with other users by using Artificial Intelligence Natural Language Understanding Skills to link with other relevant topics of discussion.  Conversations are supported across multiple media, including audio, video, speech-to-text and text-to-speech. 

Neon drives Neongecko’s AI Assistant, which captures conversations and performs next generation AI skills including polylingual conversations (for example, “Neon, speak to me in French”) and transcript recordings (for example, “Neon, email me my transcripts”), and other AI commands (for example, "Neon, play happy birthday,” “Neon, what’s the population of the United States?” or “What’s the weather in Paris?”)

Klat.com uses real-time speech-to-text processing to enable participation in multiple conversations in a single browser session and automatically shares relevant conversation segments between conversations that have converged. Conversation segments can be spoken or typed. These conversation segments might be a result of internal business conversations, business to end-user sales, business to end-user support, end-user to end-user social media, or simply connections between family and friends.

Real-time conversation transcription by Neon enables a new generation of applications, providing greater business efficiencies for: simultaneously working with multiple people and on multiple projects; interactively querying business information resources in conversations; providing options while on-hold; generating business marketing analytics from conversations; and generally enabling parallel interactions. 

Richard Leeds, Founder and President of Neongecko, said: "Klat.com enables new applications in conversation processing and conversational AI, applications that improve productivity and facilitate better social interactions, for example in multiple languages and through multiple media. Neon enables a broad range of applications, and our NeonU and NeonX AudibleAI PCs showcase conversation processing applications that improve social media and user productivity, plus they provide new AudibleAI capabilities such as audio ‘likes.’ Of course, we only reconvey recordings with our users’ permission."

Klat.com software is showcased on more than 100 public demonstration websites under the Neon beta program. Use of the latest versions of Windows, Ubuntu or MacOS iOS is recommended.

Please visit klat.com using the standard Google Chrome web browser. Links to videos and images showcasing Neon’s conversational AI skills, audible advertising coupon applications, and polylingual capabilities are available at the company website: neon.ai. Neon software source code is available on GitHub (limited license).

 

New Conversation Processing Intelligence (CPI) Tools Integrate:  

Online Conversations, Realtime Speech-to-Text, Realtime Text-to-Speech, Automated Translation, Automated Conversation Sharing, Database Lookup and AI Skills. KLAT.COM 

Device Independent API for Neon AI Applications (DIANA)

Diana provides a collection of services for automated setup and common administration tasks for business and enterprise installation of Neon AI applications. Users can choose to not run all the services. Notable uses of Diana services include:

• Diana Server TTS and Translation Services 
• Docker Support
• Kubernetes Support -- Cluster Portion Support
• Kubernetes Cluster to Cert Package -- Domain SSL Certification 
• Backend Services -- Credentials -- Cluster Management -- CM Secrets -- Docker Compose
• Ingress Support -- for Cluster Portion Support 
• Manual Configuration for Docker  
• Diana Utilities Module -- Automated Certbot and RabbitMQ Configuration, HTTPS Services, Web Socket Services, Port IO Services, and Message Bus 

Docker Compose

"Docker Compose" offers a simple method for starting a set of Diana services. All Diana services are standalone containers connected to the Neon AI RabbitMQ server. 

Diana is the starting point for Neon AI backend components and tools, many of which are user configurable. Diana generates resources for both Docker and Kubernetes. The following instructions are provided to download Neon AI Docker Containers, Diana Services and the Diana Backend Server. 

Diana commands include:

• Automatically Configure a Diana Backend
• Configure Optional HTTP Backend Services
• Generate Kubernetes Secrets Authorization Variables
• Generate a Kubernetes ConfigMap for RabbitMQ
• Start / Stop a Diana Backend

Kubernetes

For deployment, Kubernetes provides a managed solution that can handle containerization, scaling, rolling updates and more benefits. Diana deployment works with your existing clusters, managed by your system administrator using the NGINX Ingress Controller. For deploying locally, you can use MetalLB to configure a LoadBalancer.

A Diana backend can be configured automatically with the Diana configure-backend command. This will generate k8s_secret_mq-config.yml, k8s_config_rabbitmq.yml, and kubernetes.yml spec files, including creating an instance of RabbitMQ via Docker.  

Resources for Kubernetes and OpenShift can be generated for deploying containers under infrastructures using Docker Compose.

On-Device Speech To Text (STT) and Text To Speech (TTS)

Full on-device polylingual speech-to-text (STT) and text-to-speech (TTS) are provided for:

• Linux
• Raspberry Pi
• Mycroft Mark 2

In addition to English, foreign language support provided for:

• French
• Spanish
• German
• Italian
• Russian
• Polish

... and a maturing list of other languages

 

GitHub

The GitHub link for the latest default branch is: 

• https://github.com/NeonGeckoCom/neon-diana-utils/

Support

If your organization is standardizing on Neon AI, the Neon AI support team is available to help with your installation of Neon AI and Diana, and assist your IT team in using Docker Containers and Kubernetes Orchestration. 

For corporate support, please email josh@neon.ai and mention a brief reference to your application - and we'll get right back to you about helping you out. We're happy to reach out through email, GitHub or setting up a time to chat on-line, as you prefer. 

- Josh R <josh@neon.ai>