Home / Python / Deep Learning

Machine learning driven web application firewall to detect malicious queries with high accuracy.

Fwaf-Machine-Learning-driven-Web-Application-Firewall Machine learning driven web application firewall to detect malicious queries with high accuracy. Link: http://fsecurify.com/fwaf-machine-learning-driven-web-application-fire
Information
Category: Python / Deep Learning
Watchers: 40
Star: 372
Fork: 125
Last update: Jun 25, 2022
Resource links
http://fsecurify.com/fwaf-machine-learning-driven-web-application-firewall/
https://github.com/faizann24/Fwaf-Machine-Learning-driven-Web-Application-Firewall
5 Open More issues
Closed

Improved performance

I improved the performance and added some evaluation output to your script as stated in my comments on http://www.kdnuggets.com/2017/02/machine-learning-driven-firewall.html

Parameters: Using sublinear scaling, n-gram range from 1 - 3 (I used the one supplied by sklearn TfidfVectorizer), cut-off of 0.0 for TF IDF and balanced class weighting in the logistic regression

Performance Evaluation on test set: Accuracy: 0.999348 Precision: 0.983826 Recall: 0.996927 --> Missing ~150 attacks F1-Score: 0.990333 AUC: 0.999981

Opposed to original parameters: Accuracy: 0.997982 Precision: 0.996632 Recall: 0.942977 --> Missing ~2,500 attacks F1-Score: 0.969063 AUC: 0.999227

opened Mar 1, 2017 by tmechsner 1
Closed

bugs and mismatches are fixed

  • import libraries but never used.
  • no need decode function for python3
  • write tempXssQueries but there is no declaration
  • used os but didn't import
  • used set function with wrong brackets
opened Feb 15, 2017 by aysebilgegunduz 1
Closed

Modified file opening statement

Modified file opening statement to make use of with context manager. It makes sure that the file is cleanly closed after accessing the data. Also added in os.path.join because it prevents the code from breaking on different platforms which might use "" for path separation. :+1:

opened May 14, 2017 by yasoob 0
Closed

Improved performance by different class weighting

Again a small improvement in performance: I weighted the bad queries higher in the logistic regression. :-)

opened Mar 2, 2017 by tmechsner 0
Closed

fix typo and refactor

opened Feb 14, 2017 by SmartAI 0
Closed

fix typo and handle decode exception

opened Feb 14, 2017 by SmartAI 0
Open

Data Licence

Greetings, I want to use the data here for a project of mine. I need to verify the source of the data and the licence it has. Could you help me out with this?

opened Jan 6, 2022 by tentacool9 0
Open

Where do you use data from ?

Where do you use data (goodqueries and badqueries ) from ?

opened Apr 20, 2020 by hungnt31 0
Open

G

Hai

opened Jan 18, 2019 by srikanthgunja 0
Open

Script not working?

Hi, Has the ML script been run recently? I tried to run it with python 3 and 2 and got errors. For inst with python 2 (might be 2.7.15), I got the following logs: "(py27) C:\Users\Thomas Lev\Documents\Fwaf\Fwaf-Machine-Learning-driven-Web-Application-Firewall-master>python script.py C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\site-packages\sklearn\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20. "This module will be removed in 0.20.", DeprecationWarning) Traceback (most recent call last): File "script.py", line 43, in X = vectorizer.fit_transform(queries) File "C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\site-packages\sklearn\feature_extraction\text.py", line 1381, in fit_transform X = super(TfidfVectorizer, self).fit_transform(raw_documents) File "C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\site-packages\sklearn\feature_extraction\text.py", line 869, in fit_transform self.fixed_vocabulary) File "C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\site-packages\sklearn\feature_extraction\text.py", line 792, in _count_vocab for feature in analyze(doc): File "C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\site-packages\sklearn\feature_extraction\text.py", line 255, in return lambda doc: self.char_ngrams(preprocess(self.decode(doc))) File "C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\site-packages\sklearn\feature_extraction\text.py", line 116, in decode doc = doc.decode(self.encoding, self.decode_error) File "C:\Users\Thomas Lev\AppData\Local\conda\conda\envs\py27\lib\encodings\utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xc0 in position 1: invalid start byte"

Thanks

Thomas

opened Jun 1, 2018 by thlevy 2
Open

Can't run this script

When i was test,there are a lot of error codes,Can you submit the code that can run to github？

opened Feb 14, 2017 by 5up3rc 4

Contents

    adstuart Pause and restart Azure Firewall resources for optimised consumption costs in lab/testing scenarios
    fastly A Rust library for evaluating log4j substitution queries in order to determine whether or not malicious queries may exist.
    terrylinooo Web Application Firewall (WAF) for PHP.
    Ekultek Detect and bypass web application firewalls and protection systems
    neelkanthk Put malicious users, IP addresses and anonymous browser fingerprints under surveillance, log the URLs they visit and block malicious ones from accessing the Laravel app.
    3CORESec MAL-CL (Malicious Command-Line)
    sfzhang15 FaceBoxes: A CPU Real-time Face Detector with High Accuracy, IJCB, 2017
    sindresorhus Detect the indentation of code
    antonioribeiro Firewall package for Laravel applications
    firewalld Firewall daemon with D-Bus interface
    maaaaz Yet another static code analyzer for malicious Android applications
    SpiderLabs ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx
    mlinquan vue-awesome-countdown - Countdown plug-in with high performance and high accuracy for Vue 2.5.0+. Official site: https://vac.js.org
    whtsky vuex-queries - Vuex-Queries helps you write query functions in Vuex
    joesecurity Pafish Macro is a Macro enabled Office Document to detect malware analysis systems and sandboxes. It uses evasion & detection techniques implemented by malicious documents.
    lirantal Detect trojan source attacks that employ unicode bidi attacks to inject malicious code
    CoolerVoid Raptor - WAF - Web application firewall using DFA [ Current version ] - Beta
    zhuinfo An open source web application firewall component
    CSecGroup Wafid identify and fingerprint Web Application Firewall (WAF) products.
    akaunting Web Application Firewall (WAF) package for Laravel
    echo-devim Lightweight In-App Web Application Firewall for PHP
    Share this repo

    Related Repos


    Deep Learning
    837
    An implementation of chunked, compressed, N-dimensional arrays for Python.
    zarr-developers Zarr Latest Release Package Status License Build Status Coverage Downloads Gitter Citation What is it? Zarr is a Python package providing an implement
     
    Deep Learning
    317
    Serverless proxy for Spark cluster
    Hydrospheredata Hydrosphere Mist Hydrosphere Mist is a serverless proxy for Spark cluster. Mist provides a new functional programming framework and deployment model f
     
    Deep Learning
    2.5k
    Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
    intel-analytics Distributed TensorFlow, PyTorch, Keras and BigDL on Apache Spark & Ray Analytics Zoo is an open source Big Data AI platform, and includes the followin
     
    Deep Learning
    10
    Black-Scholes StarkNet Library
    araghava Cairo Black-Scholes Library Black-Scholes library implemented as a Cairo smart contract. All inputs, outputs, and internal calculations use 27-digit f
     
    Deep Learning
    5
    Motoko library to help create an append-only logger actor
    ninegua IC Logger This motoko library provides a module to help create an append-only logger actor. Usage You can use this library with the vessel package man
     
    Deep Learning
    4
    LeetCode_Solution
    ashishpatel26 LeetCode_Solution LeetCode: https://leetcode.com/problemset/all/ # Title Difficulty Colab Code 1 Two Sum Easy 2 Add Two Numbers Medium 3 Longest Subst
     
    Deep Learning
    4
    OCR-D wrapper for detectron2 based segmentation models
    bertsky ocrd_detectron2 OCR-D wrapper for detectron2 based segmentation models Introduction Installation Usage OCR-D processor interface ocrd-detectron2-segm
     
    Deep Learning
    134
    Check for LDAP protections regarding the relay of NTLM authentication
    zyn3rgy LDAP Relay Scan A tool to check Domain Controllers for LDAP server protections regarding the relay of NTLM authentication. If you're interested in the
     
    Deep Learning
    66
    Fuzzware's main repository. Start here to install.
    fuzzware-fuzzer Fuzzware Fuzzware is a project for automated, self-configuring fuzzing of firmware images. The idea of this project is to configure the memory ranges
     
    Deep Learning
    35
    Time Weighted Asset Mints
    abigger87 TWAM • Time Weighted Asset Mints A minting harness enabling time-weighted assets to determine minting prices. How it works Requirements: maxMintingAmo
     
    Deep Learning
    33
    A complete, self-contained example for training ImageNet at state-of-the-art speed with FF...
    libffcv ffcv ImageNet Training A minimal, single-file PyTorch ImageNet training script designed for hackability. Run train_imagenet.py to get... ...high accur
     
    Deep Learning
    33
    เอกสารประกอบการสอนทุกเนื้อหาในช่องยูทูป KongRuksiam Official
    kongruksiamza 📖 เอกสารประกอบการสอน (PDF) 🔏 Blockchain & Smart Contract Blockchain เบื้องต้น Smart Contract & Solidity เบื้องต้น 💻 Programming เขียนโปรแกรมภาษา Ja
     
    Deep Learning
    34
    Omnivore A Single Model for Many Visual Modalities
    facebookresearch Omnivore: A Single Model for Many Visual Modalities [paper][website] OMNIVORE is a single vision model for many different visual modalities. It learns
     
    Deep Learning
    30
    Python implementation of the Lox language from Robert Nystrom's Crafting Interpreters
    dabeaz pylox Python implementation of the Lox language from Robert Nystrom's Crafting Interpreters. https://craftinginterpreters.com. This only implements th
     
    Deep Learning
    29
    ParaGen is a PyTorch deep learning framework for parallel sequence generation.
    bytedance ParaGen ParaGen is a PyTorch deep learning framework for parallel sequence generation. Apart from sequence generation, ParaGen also enhances various N