Home / Python / Deep Learning

Supervised learning for novelty detection in text

Pythia Novelty detection in text corpora Pythia is Lab41's exploration of approaches to novel content detection. We are interested in making it easier to tell when a document coming into a corpus has something ne
Information
Category: Python / Deep Learning
Watchers: 22
Star: 80
Fork: 24
Last update: Jan 4, 2022
Resource links
https://github.com/Lab41/pythia

Pythia

pythia logo

CircleCI codecov Docker Automated build

Novelty detection in text corpora

Pythia is Lab41's exploration of approaches to novel content detection. We are interested in making it easier to tell when a document coming into a corpus has something new to say. We welcome your contributions (see our contributor guidelines) and attention.

Run a quick experiment

You can get started very quickly on a system with Docker using the following commands to pull our publicly available image and train an XGBoost model on the sample data that comes with the repository:

docker pull lab41/pythia
docker run -it lab41/pythia experiments/experiments.py with XGB=True BOW_APPEND=True BOW_PRODUCT=True

Tests and building

docker build -t lab41/pythia .     # runs tests and builds project image

Prerequisites

Our code is written in Python 3. It requires a recent version of Anaconda, as well as a C/C++ compiler system, e.g. GNU gcc/g++ (available in package build-essential on Ubuntu/Debian systems).

Once these have been installed on your system, envs/make_envs.sh will install the necessary Python dependencies in an Anaconda environment called py3-pythia.

The Docker-based distribution comes prepackaged with all necessary dependencies, provided Docker itself is available.

Documentation

Prebuilt documentation available at http://lab41.github.io/pythia

Contents
    alinlab CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
    PoplarZhaoYang A semi-supervised neural network
    SXKDZ A curated list for awesome self-supervised learning for graphs.
    Bartzi Code for the paper STN-OCR: A single Neural Network for Text Detection and Text Recognition
    zhenyingfang Temporal Action Detection & Weakly Supervised Temporal Action Detection & Temporal Action Proposal Generation
    XgDuan Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.
    philferriere Weakly Supervised Segmentation with Tensorflow. Implements instance segmentation as described in Simple Does It: Weakly Supervised Instance and Semantic Segmentation, by Khoreva et al. (CVPR 2017).
    qizhuli Weakly- and Semi-Supervised Panoptic Segmentation (ECCV18)
    justin941208 MatchGAN: A Self-supervised Semi-supervised Conditional Generative Adversarial Network
    zlai0 Self-supervised Learning for Video Correspondence Flow (BMVC 2019)
    eragonruan text detection mainly based on ctpn model in tensorflow, id card detect, connectionist text proposal network
    uizard-technologies Code for the paper: RealMix: Towards Realistic Semi-Supervised Deep Learning Algorithms
    HobbitLong PyTorch implementation of "Supervised Contrastive Learning" (and SimCLR incidentally)
    UMBCvision Backdoor Attacks on Self-Supervised Learning
    mkocabas Self-Supervised Learning of 3D Human Pose using Multi-view Geometry (CVPR2019)
    afourast Implementation for ECCV20 paper "Self-Supervised Learning of audio-visual objects from video"
    Namkyeong An official source code for "Augmentation-Free Self-Supervised Learning on Graphs"
    FriendsOfREDAXO YCOM Impersonate. Login as selected YCOM user 🧙‍♂️in frontend.
    Joker316701882 This is tensorflow implementation for cvpr2017 paper "Deeply Supervised Salient Object Detection with Short Connections"
    AdneneBoumessouer This project proposes an end-to-end framework for semi-supervised Anomaly Detection and Segmentation in images based on Deep Learning.
    wonheeML Multi-task Self-supervised Object Detection via Recycling of Bounding Box Annotations (CVPR, 2019)
    Share this repo

    Related Repos


    Deep Learning
    634
    Bagua Speeds up PyTorch
    BaguaSys Bagua is a deep learning training acceleration framework for PyTorch developed by AI [email protected] Technology and DS3 [email protected] Zürich. Bagua curren
     
    Deep Learning
    4.6k
    Distributed Evolutionary Algorithms in Python
    DEAP DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru
     
    Deep Learning
    85
    Python library for converting Apache Spark ML pipelines to PMML
    jpmml PySpark2PMML Python library for converting Apache Spark ML pipelines to PMML. Features This package provides Python wrapper classes and functions for
     
    Deep Learning
    5
    R library for converting Apache Spark ML pipelines to PMML
    jpmml Sparklyr2PMML R library for converting Apache Spark ML pipelines to PMML. Features This package provides R wrapper classes and functions for the JPMML
     
    Deep Learning
    5.7k
    H2O is an in-memory platform for distributed, scalable machine learning
    h2oai H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
     
    Deep Learning
    144
    Optimised tools for group-indexing operations: aggregated sum and more
    ml31415 numpy-groupies This package consists of a small library of optimised tools for doing things that can roughly be considered "group-indexing operations"
     
    Deep Learning
    6.9k
    Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and explorat...
    vaexio What is Vaex? Vaex is a high performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular data
     
    Deep Learning
    2.1k
    Deequ is a library built on top of Apache Spark for defining "unit tests for data", which ...
    awslabs Deequ - Unit Tests for Data Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large da
     
    Deep Learning
    409
    Deep Anomaly Detection with Outlier Exposure (ICLR 2019)
    hendrycks Outlier Exposure This repository contains the essential code for the paper Deep Anomaly Detection with Outlier Exposure (ICLR 2019). Requires Python 3
     
    Deep Learning
    303
    (MLSys' 21) An Acceleration System for Large-scare Unsupervised Heterogeneous Outlier Dete...
    yzhao062 SUOD: Accelerating Large-scare Unsupervised Heterogeneous Outlier Detection Deployment & Documentation & Stats Build Status & Coverage & Maintainabili
     
    Deep Learning
    597
    Library for exploring and validating machine learning data
    tensorflow TensorFlow Data Validation TensorFlow Data Validation (TFDV) is a library for exploring and validating machine learning data. It is designed to be hig
     
    Deep Learning
    1.6k
    Automated machine learning for production and analytics
    ClimbsRocks auto_ml Automated machine learning for production and analytics Installation pip install auto_ml Getting started from auto_ml import Predictor from au
     
    Deep Learning
    673
    Use evolutionary algorithms instead of gridsearch in scikit-learn
    rsteca sklearn-deap Use evolutionary algorithms instead of gridsearch in scikit-learn. This allows you to reduce the time required to find the best parameter
     
    Deep Learning
    20
    Naive Bayesian Classifier written in APL
    mattcunningham Naive Bayesian Classifier in APL © 2015-2016 Matthew Cunningham About This is a simple naive bayesian classifier to gain independent probabilistic ass
     
    Deep Learning
    10
    Conjunto de scripts para treinar um Sistema de Recomendação Híbrido baseado nos algoritmos...
    SeniorSA hybrid-rs-trainner Treine suas engines de recomendação with zero code! :) Sumário Sistemas de Recomendação Collaborative Filtering Content-Based Filte