Supervised learning for novelty detection in text

README
Issues 6

Pythia

Novelty detection in text corpora

Pythia is Lab41's exploration of approaches to novel content detection. We are interested in making it easier to tell when a document coming into a corpus has something new to say. We welcome your contributions (see our contributor guidelines) and attention.

Run a quick experiment

You can get started very quickly on a system with Docker using the following commands to pull our publicly available image and train an XGBoost model on the sample data that comes with the repository:

docker pull lab41/pythia
docker run -it lab41/pythia experiments/experiments.py with XGB=True BOW_APPEND=True BOW_PRODUCT=True

Tests and building

docker build -t lab41/pythia .     # runs tests and builds project image

Prerequisites

Our code is written in Python 3. It requires a recent version of Anaconda, as well as a C/C++ compiler system, e.g. GNU gcc/g++ (available in package build-essential on Ubuntu/Debian systems).

Once these have been installed on your system, envs/make_envs.sh will install the necessary Python dependencies in an Anaconda environment called py3-pythia.

The Docker-based distribution comes prepackaged with all necessary dependencies, provided Docker itself is available.

Documentation

Prebuilt documentation available at http://lab41.github.io/pythia

0 Open More issues

Closed

Type and expected format of some featurization techniques unclear

opened Sep 30, 2016 by pcallier 6

Closed

Add json schemas for classifier, evaluation modules

These schemas should be considered drafts and up for discussion.

opened Jun 14, 2016 by pcallier 5

Closed

Write test cases for pipeline

End-to-end analysis scripts with deterministic behavior should probably come with tests attached (they have already broken in the week they've been up). We will all write some.

opened Jun 1, 2016 by pcallier 4

Closed

Make Sacred log full results, data directory, classifier parameters when using hyperopt

When using hyperopt, Sacred does not receive:

the full results dict as logged in experiments.py
the data directory, which we use to identify the data source and it logged classifier parameters in an idiosyncratic way. This PR addresses all three issues, so that Sacred logs the same results for Hyperopt-controlled and other experiments (and Hyperopt still sees only -F1), the data directory is stored alongside other run hyperparameters, and classifier hyperparameters are logged as in other experiments. This will make it possible to compare results from Hyperopt-controlled experiments to other experiments.

opened Sep 17, 2016 by pcallier 3

Closed

Memory problems

BOW_APPEND + XGB, on large_sites, fails with the following error:

ERROR - pythia_experiment - Failed after 0:29:42!
Traceback (most recent calls WITHOUT Sacred internals):
  File "experiments/experiments.py", line 260, in run_experiment
    SEED)
  File "/pythia/src/pipelines/master_pipeline.py", line 67, in main
    predicted_labels, perform_results = main_mem_net.test_mem_network(mem_net_model, model_name, **algorithms['mem_net'])
  File "/pythia/src/pipelines/xgb.py", line 40, in main
    xclassifier = run_model(train_data, train_target, **args_dict)
  File "/pythia/src/pipelines/xgb.py", line 24, in run_model
    xclassifier.fit(train_data, train_labels)
  File "/opt/conda/envs/py3-pythia/lib/python3.5/site-packages/xgboost/sklearn.py", line 337, in fit
    missing=self.missing)
  File "/opt/conda/envs/py3-pythia/lib/python3.5/site-packages/xgboost/core.py", line 232, in __init__
    self._init_from_npy2d(data, missing)
  File "/opt/conda/envs/py3-pythia/lib/python3.5/site-packages/xgboost/core.py", line 284, in _init_from_npy2d
    ctypes.byref(self.handle)))
  File "/opt/conda/envs/py3-pythia/lib/python3.5/site-packages/xgboost/core.py", line 97, in _check_call
    raise XGBoostError(_LIB.XGBGetLastError())
xgboost.core.XGBoostError: b'std::bad_alloc'

Seems to indicate running out of memory?

opened Sep 6, 2016 by pcallier 3

Closed