Intent tools

Tools for processing intents from RASA JSONs to Facebook fast-text learning sets.

Because there is good GUI: Rasa NLU Trainer

CLI Usage

• Convert RASA json to fast-text learning set

$ intools jsonToText ./testData.json ./testData.txt

• Convert RASA json to fast-text learning set and multiply by entities

$ intools jsonToText -m ./testData.json ./testData.txt

• Make word vectors learning set from wiki XML export

$ intools wikiToText ./testData.xml ./testData.txt

API

Classes

MultiplicatorStream

MultiplicatorStream

Pipeline

Functions

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

MultiplicatorStream

Kind: global class

• MultiplicatorStream
  • new MultiplicatorStream()
  • new MultiplicatorStream(getVariants)

new MultiplicatorStream()

Multiplicates a learning set data with available entities information

Example

const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');
 
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
 
const entities = new main.EntitiesFromJson(from);
 
const pipeline = [
    new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];
 
entities.loadEntities()
    .then(() => main.jsonToText(from, to, pipeline))
    .catch(e => console.error(e));

new MultiplicatorStream(getVariants)

MultiplicatorStream

Kind: global class

• MultiplicatorStream
  • new MultiplicatorStream()
  • new MultiplicatorStream(getVariants)

new MultiplicatorStream()

Multiplicates a learning set data with available entities information

Example

const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');
 
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
 
const entities = new main.EntitiesFromJson(from);
 
const pipeline = [
    new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];
 
entities.loadEntities()
    .then(() => main.jsonToText(from, to, pipeline))
    .catch(e => console.error(e));

new MultiplicatorStream(getVariants)

Pipeline

Kind: global class

• Pipeline
  • new Pipeline()
  • .add(pipe) ⇒ this
  • .promise() ⇒ promise

new Pipeline()

Simple tool, which creates a Promise from pipeline of streams

pipeline.add(pipe) ⇒ this

Append a stream

Kind: instance method of Pipeline

pipeline.promise() ⇒ promise

Get a promise

Kind: instance method of Pipeline

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

Kind: global function

Example

const path = require('path');
const { jsonToText } = require('intent-tools');
 
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
 
main.jsonToText(from, to)
    .catch(e => console.error(e));

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

Kind: global function

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

Kind: global function Returs: string