Blazingly fast in memory DataFrames in Rust

Polars is a DataFrames library implemented in Rust, using Apache Arrow as backend. It's focus is being a fast in memory DataFrame library that only support core functionality.

Information

Category: Rust / Data processing
Watchers: 72
Star: 5.7k
Fork: 325
Last update: May 8, 2022

Resource links

https://github.com/pola-rs/polars

Polars (WIP)

Blazingly fast in memory DataFrames in Rust

Polars is a DataFrames library implemented in Rust, using Apache Arrow as backend. It's focus is being a fast in memory DataFrame library that only support core functionality.

Performance

GroupBy

Functionality

Series

DataFrame

Data types

Examples

use polars::prelude::*;

// Create first df.
let s0 = Series::init("days", [0, 1, 2, 3, 4].as_ref());
let s1 = Series::init("temp", [22.1, 19.9, 7., 2., 3.].as_ref());
let temp = DataFrame::new_from_columns(vec![s0, s1]).unwrap();

// Create second df.
let s0 = Series::init("days", [1, 2].as_ref());
let s1 = Series::init("rain", [0.1, 0.2].as_ref());
let rain = DataFrame::new_from_columns(vec![s0, s1]).unwrap();

// Left join on days column.
let joined = temp.left_join(&rain, "days", "days");
println!("{}", joined.unwrap())

           days           temp           rain
            i32            f64            f64
            ---            ---            ---

              0           22.1           null
              1           19.9            0.1
              2              7            0.2
              3              2           null
              4              3           null

Arithmetic

use polars::prelude::*;

let s: Series = [1, 2, 3].iter().collect(); 
let s_squared = &s * &s;

Custom functions

use polars::prelude::*;

let s: Series = [1, 2, 3].iter().collect(); 
let s_squared: Series = s.i32()
     .expect("datatype mismatch")
     .into_iter()
     .map(|optional_v| {
         match optional_v {
             Some(v) => Some(v * v),
             None => None, // null value
         }
 }).collect();

143 Open More issues

Closed

Groupby on integer + date column of large dataframe requests enormous memory alloc [Windows only]

Are you using Python or Rust?

Python

Which feature gates did you use?

What version of polars are you using?

0.10.5

What operating system are you using polars on?

Windows10

Describe your bug.

Polarsa fails to execute groupby on modestly sized dataframe

What are the steps to reproduce the behavior?

I've used something like the below code to generate the data (it's pretty big) and then run the polars_load function

import pandas as pd
import polars as pl

def make_some_data():
  n_ids = int(1e3)
  n_features = 10
  freq = "1H"
  year = 2000
  start = f"{year}0101"
  end = f"{year + 1}0615" # one year and a bit
  date_index = pd.date_range(start=start, end=end, freq=freq, closed="left")
  dates= np.tile(date_index, n_ids)
  n_dates = len(date_index)
  
  ids = np.repeat(np.arange(n_ids, dtype=np.int32), n_dates)
  features= np.random.randn(n_ids * n_dates, n_features).astype(np.float32)
  df = pd.DataFrame(
      {
          "ids": ids,
          "dates": dates,
      }
  )
  df[[f"feature_{i}" for i in range(n_features)]] = features
  
  print(f"\t[year={year}] n_ids={n_ids}, n_dates={n_dates}, n_rows={n_ids * n_dates}, n_cols={n_features}", flush=True)
  df.to_parquet(data_path / f"features_{year}.parquet")

def pandas_load(data_dir: Path):
    import pandas as pd

    df = pd.read_parquet(f)
    agg_df = df.groupby(["ids", "dates"]).agg("mean")
    return agg_df


def polars_load(df):
    df = pl.read_parquet(f)
    agg_df = df.groupby(["ids", "dates"]).agg(pl.col("*").mean())

    return agg_df

If we cannot reproduce the bug, it is unlikely that we will be able fix it.

What is the actual behavior?

I get the following error

UserWarning: Conversion of (potentially) timezone aware to naive datetimes. TZ information may be 
lost
  "Conversion of (potentially) timezone aware to naive datetimes. TZ information may be lost",
memory allocation of memory allocation of memory allocation of 191160001911600019116000 bytes failed

The CPU usage goes to 100, memory stays modest, and then the screen goes dark and I regain control after a minute or two with an unkillable python.exe -System Error window

What is the expected behavior?

Running the pandas_load function exhibits the correct behaviour.

help wanted

opened Oct 19, 2021 by CHDev93 43

Closed

Failed join #2

Are you using Python or Rust?

Python

What version of polars are you using?

0.12.18

What operating system are you using polars on?

CentOS Linux release 8.1.1911 (Core)

Describe your bug.

Rows aren't joining that should. I.e. on an inner join, some rows are being omitted from the joined data frame that should be there. On a left join, some of the right columns are filled with null values when they shouldn't be.

What are the steps to reproduce the behavior?

Output:

>>> ./reproduce_error.py
--right_df--
shape: (1, 4)
┌──────────┬───────────────────┬───────────────────┬────────────────┐
│ join_col ┆ new_col_from_join ┆ table_1_indicator ┆ other_join_len │
│ ---      ┆ ---               ┆ ---               ┆ ---            │
│ i64      ┆ f64               ┆ bool              ┆ i64            │
╞══════════╪═══════════════════╪═══════════════════╪════════════════╡
│ 59546483 ┆ 6.9900e-46        ┆ true              ┆ null           │
└──────────┴───────────────────┴───────────────────┴────────────────┘
--left_df--
shape: (1, 3)
┌────────────┬──────────┬────────────────┐
│ temp       ┆ join_col ┆ other_join_len │
│ ---        ┆ ---      ┆ ---            │
│ str        ┆ i64      ┆ i64            │
╞════════════╪══════════╪════════════════╡
│ 1_59546483 ┆ 59546483 ┆ null           │
└────────────┴──────────┴────────────────┘
--join--
shape: (1, 5)
┌────────────┬──────────┬────────────────┬───────────────────┬───────────────────┐
│ temp       ┆ join_col ┆ other_join_len ┆ new_col_from_join ┆ table_1_indicator │
│ ---        ┆ ---      ┆ ---            ┆ ---               ┆ ---               │
│ str        ┆ i64      ┆ i64            ┆ f64               ┆ bool              │
╞════════════╪══════════╪════════════════╪═══════════════════╪═══════════════════╡
│ 1_59546483 ┆ 59546483 ┆ null           ┆ null              ┆ null              │
└────────────┴──────────┴────────────────┴───────────────────┴───────────────────┘

Notice that the two columns being joined on ('join_col' and 'other_join_len') are the same in the right and left DFs, but the right columns don't show up in the joined DF.

Reproducing code

#!/usr/bin/env python3

import polars as pl

right_df_1 = pl.scan_csv(
    'repro_tsv_1.txt',
    sep='\t',
).select([
    'join_col',
    'new_col_from_join',
    pl.lit(True).alias('table_1_indicator'),
    pl.lit(None).cast(int).alias('other_join_len'),
])

right_df_2 = pl.scan_csv(
    'repro_tsv_2.txt',
    sep='\t',
).select([
    'join_col',
    'new_col_from_join',
    pl.col('other_join_col').str.lengths().cast(int).alias('other_join_len'),
]).groupby(['join_col', 'other_join_len']).agg([
    pl.col('new_col_from_join').min().alias('new_col_from_join'),
]).with_columns([
    pl.lit(False).alias('table_1_indicator')
]).select([ 'join_col', 'new_col_from_join', 'table_1_indicator', 'other_join_len'])

right_df = pl.concat([right_df_1, right_df_2])
print('--right_df--')
print(right_df.filter((pl.col('join_col') == 59546483) & pl.col('other_join_len').is_null()).collect())

rownames_fname = 'munged_rownames.txt'
with open(rownames_fname) as var_file:
    rownames = [line.strip() for line in var_file if line.strip()]

left_df = pl.DataFrame({
    'temp': rownames,
}).lazy().with_columns([
    pl.col('temp').str.extract('^[^_]*_([^_]*)', 1).cast(int).alias('join_col'),
    pl.col('temp').str.extract('^[^_]*_[^_]*_([^_]*)$', 1).str.lengths().cast(int).alias('other_join_len'),
])
print('--left_df--')
print(left_df.filter((pl.col('join_col') == 59546483) & pl.col('other_join_len').is_null()).collect())

total_df = left_df.join(
    right_df,
    how='left',
    on=['join_col', 'other_join_len']
).collect()
print('--join--')
print(total_df.filter((pl.col('join_col') == 59546483) & pl.col('other_join_len').is_null()))

repro_tsv_1.txt repro_tsv_2.txt munged_rownames.txt

bug

opened Jan 30, 2022 by LiterallyUniqueLogin 32

Closed

shift_and_fill by groups + other operations by grouping variables

I'm new to polars and would like to understand how to run certain methods by group variables

A few methods include: lags and rolling stats with varying window sizes: lag1, lag2, ..., lagN ma1, ma2, ..., maN

Using python

# Create data (data from apply section of docs)
data = pl.DataFrame(
    {
        "A": [1, 2, 3, 4, 5],
        "fruits": ["banana", "banana", "apple", "apple", "banana"],
        "B": [5, 4, 3, 2, 1],
        "cars": ["beetle", "audi", "beetle", "beetle", "beetle"],
    }
)

# Define variables
ShiftColName = "B"
N = 1
ImputeValue = -1
GroupVariable = "fruits"

# shift_and_fill: no groups
data.hstack(data[[ShiftColName]].shift_and_fill(N, fill_value = ImputeValue).rename({ShiftColName : 'Lag_' + str(N) + '_' + ShiftColName}))

# shift_and_fill: by fruits?
?

opened Aug 10, 2021 by AdrianAntico 26

Closed

Add optional lexical ordering of dtype `Categorical`

Are you using Python or Rust?

Python

What version of polars are you using?

0.12.20

What operating system are you using polars on?

macOS Monterey

Describe your bug.

DataFrame.sort() on Categorical type behave incorrectly.

What are the steps to reproduce the behavior?

df = pl.DataFrame(
    [
        pl.Series("col1", [1, 1], dtype=pl.UInt8),
        pl.Series("col2", ["foo", "bar"], dtype=pl.Categorical),
        pl.Series("col3", [3.3, 1.1], dtype=pl.Float64),
    ]
)
df.sort(['col1', 'col2'])

What is the actual behavior?

shape: (2, 3)
┌──────┬──────┬──────┐
│ col1 ┆ col2 ┆ col3 │
│ ---  ┆ ---  ┆ ---  │
│ u8   ┆ cat  ┆ f64  │
╞══════╪══════╪══════╡
│ 1    ┆ foo  ┆ 3.3  │
├╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤
│ 1    ┆ bar  ┆ 1.1  │
└──────┴──────┴──────┘

What is the expected behavior?

shape: (2, 3)
┌──────┬──────┬──────┐
│ col1 ┆ col2 ┆ col3 │
│ ---  ┆ ---  ┆ ---  │
│ u8   ┆ cat  ┆ f64  │
╞══════╪══════╪══════╡
│ 1    ┆ bar  ┆ 1.1  │
├╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤
│ 1    ┆ foo  ┆ 3.3  │
└──────┴──────┴──────┘

I think polars should sort correctly on categorical type, like the following:

df.sort('col2')
shape: (2, 3)
┌──────┬──────┬──────┐
│ col1 ┆ col2 ┆ col3 │
│ ---  ┆ ---  ┆ ---  │
│ u8   ┆ cat  ┆ f64  │
╞══════╪══════╪══════╡
│ 1    ┆ bar  ┆ 1.1  │
├╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤
│ 1    ┆ foo  ┆ 3.3  │
└──────┴──────┴──────┘

feature

opened Feb 6, 2022 by mutecamel 25

Closed

Missing arm build for linux

Are you using Python or Rust?

Python

What version of polars are you using?

latest, 0.13.3

What operating system are you using polars on?

Debian 10

Describe your bug.

I'm trying to install polars in a python docker container, where rust/cargo is not installed.

We use these docker images regularly at work. During pip install polars, it raises the following error, that it requires rust. Is this correct? From the github readme, I do not get the impression that rust is a dependency, though I understand it is built using rust. My colleague was very surprised when I told him, and suggested I post here.

Error:

 > [3/3] RUN pip install polars==0.13.3:
#5 1.787 Collecting polars==0.13.3
#5 1.909   Downloading polars-0.13.3.tar.gz (729 kB)
#5 2.543   Installing build dependencies: started
#5 19.12   Installing build dependencies: finished with status 'done'
#5 19.12   Getting requirements to build wheel: started
#5 19.16   Getting requirements to build wheel: finished with status 'done'
#5 19.16     Preparing wheel metadata: started
#5 19.19     Preparing wheel metadata: finished with status 'error'
#5 19.19     ERROR: Command errored out with exit status 1:
#5 19.19      command: /usr/local/bin/python /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp_jualw5a
#5 19.19          cwd: /tmp/pip-install-mpfc2sos/polars_a0e8fca0fa6e478fa5565d66ba22b23f
#5 19.19     Complete output (6 lines):
#5 19.19
#5 19.19     Cargo, the Rust package manager, is not installed or is not on PATH.
#5 19.19     This package requires Rust and Cargo to compile extensions. Install it through
#5 19.19     the system's package manager or via https://rustup.rs/
#5 19.19
#5 19.19     Checking for Rust toolchain....

What are the steps to reproduce the behavior?

Save the following file as Dockerfile in a directory on your computer.

FROM python:3.9-slim-buster
WORKDIR /usr/src/app
RUN pip install polars==0.13.3

From the above directory, call docker build -t test-polars .

What is the actual behavior?

Error, as above

What is the expected behavior?

Either it should be made clear that rust is a dependency, or it should install without error :)

help wanted

opened Feb 16, 2022 by thomasaarholt 22

Closed

How to get the correct date with SampleRule::Day(n)

Hi @ritchie46 In my case, I use SampleRule::Day() to get the DataFrame date, but I can't get the correct date.

my csv file data:

TESTSYMBOL,20210319 23:58:49.724418,0.9145,0.95476,1
TESTSYMBOL,20210319 23:58:49.780350,0.9545,0.95476,1
TESTSYMBOL,20210319 23:58:49.826159,0.9545,0.95476,1
TESTSYMBOL,20210319 23:58:49.860344,0.95449,0.95476,1
TESTSYMBOL,20210319 23:58:50.163641,0.95449,0.9547,1
TESTSYMBOL,20210319 23:58:50.186391,0.95447,0.95469,10
TESTSYMBOL,20210319 23:58:50.238856,0.95449,0.95472,1
TESTSYMBOL,20210319 23:58:50.290935,0.95443,0.95467,1
TESTSYMBOL,20210319 23:58:50.341803,0.95443,0.95467,1

my test code:

     rule = SampleRule::Day(1);
    let df_ohlc = df.downsample("u", rule).unwrap()
        .agg(&[("c", &["first", "max", "min", "last"])]).unwrap()
        .sort("u", false).unwrap();
    debug!("Symbol = {}, period type = {}, ohlc === {:?}", value.s, value.t, df_ohlc);

the test result:

 rule = SampleRule::Day(1)  -->  2021-03-19 00:00:00   //correct
 rule = SampleRule::Day(2)  -->  2021-03-18 00:00:00   //correct
 rule = SampleRule::Day(3)  -->  2021-03-19 00:00:00   //not correct, should be 2021-03-17 00:00:00
 rule = SampleRule::Day(4)  -->  2021-03-18 00:00:00   //not correct, should be 2021-03-16 00:00:00
 rule = SampleRule::Day(5)  -->  2021-03-19 00:00:00   //not correct, should be 2021-03-15 00:00:00
 rule = SampleRule::Day(6)  -->  2021-03-16 00:00:00   //not correct, should be 2021-03-14 00:00:00
 rule = SampleRule::Day(7)  -->  2021-03-18 00:00:00   //not correct, should be 2021-03-13 00:00:00

Only SampleRule::Day(1) and SampleRule::Day(2) are correct, all others are errors. So how to use the SampleRule::Day() in a correct way? Thank you very much?

opened Mar 24, 2021 by CHCP 22

Open

Polars serialize and deserialize error

What language are you using?

Python

Which feature gates did you use?

This can be ignored by Python & JS users.

Have you tried latest version of polars?

[yes]

If the problem was resolved, please update polars. :)

What version of polars are you using?

0.13.31

What operating system are you using polars on?

Linux

What language version are you using

python 3.9

Describe your bug.

Serialize and deserialize like below, not possible in polars >= "0.13.6"

import polars as pl

pl.read_json(
    pl.DataFrame({"A": [1, 2, 3, 4], "B": [5, 6, 7, 8]}).to_json()
)

Result in FileNotFoundError: No such file or directory (os error 2)

What are the steps to reproduce the behavior?

see codeblock above.

What is the actual behavior?

see codeblock above.

What is the expected behavior?

in polars <= "0.13.5" the JSON object is read as a polars DataFrame.

bug

opened May 12, 2022 by RDWimmers 2

Open

Support unicode as delimiter when reading the CSV file. For example a CSV delimiter could be §

Describe your feature request

I have a CSV file with a delimiter of §

The reason to choose a § as the delimiter in our project is that, the log message itself could have lots of "," in the log fields. To avoid this collision we have chosen a §, a rarely used Unicode char as the delimiter.

I use the following code to read it with python.

import polars as pl 
df = pl.read_csv(r"path/to/test.log", sep="§")

It is expected to read the CSV file successfully. But, it reports errors

~\Anaconda3\lib\site-packages\polars\functions.py in read_csv(file, infer_schema_length, batch_size, has_headers, ignore_errors, stop_after_n_rows, skip_rows, projection, sep, columns, rechunk, encoding, n_threads, dtype, new_columns, use_pyarrow, low_memory, comment_char, storage_options, null_values)
    257                     skip_rows=skip_rows, autogenerate_column_names=not has_headers
    258                 ),
--> 259                 pa.csv.ParseOptions(delimiter=sep),
    260                 pa.csv.ConvertOptions(
    261                     column_types=None,

~\Anaconda3\lib\site-packages\pyarrow\_csv.pyx in pyarrow._csv.ParseOptions.__init__()

~\Anaconda3\lib\site-packages\pyarrow\_csv.pyx in pyarrow._csv.ParseOptions.delimiter.__set__()

~\Anaconda3\lib\site-packages\pyarrow\_csv.pyx in pyarrow._csv._single_char()

ValueError: Expecting an ASCII character

With the same file, the pandas can open it successfully.

import pandas as pd
df = pd.read_csv(r"path/to/test.log", sep="§", encoding='utf8', engine='python')

Suggest supporting the Unicode delimiters.

The test.log content is listed below with one header and one log entry.

sequenceid§time§logger§level§processid§processname§threadid§threadname§machinename§message§properties
104880§2018-10-01 00:00:12.6162§Services.Monitoring.SystemPerformanceMonitor§Debug§11752§Services.Logging§27§§DESKTOP-D5NFLAJ§"Total CPU usage = 46.85375%, Free RAM = 8.352539 GB, Page File Usage = 0%  Pages swapped/disk: 0.6660985 [Pages/sec]
process use in properties."§chrome 13332∈CPU:   0%   RAM:     21 MB  Threads:   13⊙chrome 13352∈CPU:   0%   RAM:      8 MB  Threads:    2

feature

opened May 11, 2022 by ForrestFeng 1

Open

Typed Series for json_path_match

Nested JSON fields are unfortunately common in some parquet and JSON files. While polars provides a json_path_match function, it can only return a string in the current implementation. This makes it of limited value when working with nested JSON fields of various types within a Utf8Chunked array. I did a POC locally showing that correct typing can be achieved with a few tweaks to json_path_match, and a modification of the deserialization approach in arrow2. I put up a PR on arrow2 (https://github.com/jorgecarleitao/arrow2/pull/989) to address the underlying support for JSON parsing from a arrow2::array::Utf8Array, and will follow up with a PR that uses that support to output a Series with proper type.

When this is added, I think the API for json_path_match should change as follows:

fn json_path_match(&self, json_path: &str) -> Result<Series> {
    ...
}

feature

opened May 11, 2022 by cjermain 1

Open

DataFrame.pivot() with empty list of index columns

Describe your feature request

I think it would be helpful if DataFrame.pivot() could be called with an empty list of index columns. In my application the list of index columns is not known in advance, and may or may not be empty.

If no index columns are specified, the rows could be grouped only by the columns specified in the "columns" argument. The result would be the same as can currently be achieved using a temporary dummy column, as shown below:

import polars as pl
df = pl.DataFrame({'a': ['one', 'two', 'one', 'two'], 'b': [1, 2, 3, 4]})
df.with_column(pl.lit(0).alias('dummy')).pivot(columns='a', values='b', index='dummy', aggregate_fn='mean').drop('dummy')

┌─────┬─────┐
│ one ┆ two │
│ --- ┆ --- │
│ f64 ┆ f64 │
╞═════╪═════╡
│ 2.0 ┆ 3.0 │
└─────┴─────┘

feature

opened May 11, 2022 by adamgreg 0

Open

concat error

What language are you using?

Python

Have you tried latest version of polars?

yes

What version of polars are you using?

0.13.31

What operating system are you using polars on?

windows11

What language version are you using

python 3.9

Describe your bug.

when use concat, if one of dataframe if empty , an issue will raise.

bug

opened May 11, 2022 by coolstudio1678 2

Open

Trim leading and trailing whitespaces while parsing csv data

I often find myself in the situation where I want to parse csv data which have been vertically aligned in the text file (because it makes the files prettier thus easier to read in terminal). However, these leading and trailing spaces come in the way of properly parsing numeric values with polars. This is why I propose to add a switch that lets us configure a way to drop these leading and trailing spaces.

Example: Parsing the following data would not work before edit but it could be accepted with this PR

INSTANCE     | STATUS | UB   | LB                   | DURATION | PEAK-MEMORY
05items/005 | Proved   | 1471 | 1470.998622 | 0                 | 0.004846898839
05items/033 | Proved   | 1104 | 1103.999        | 0                 | 0.004846531898
05items/024 | Proved   | 1308 | 1307.998815 | 0                 | 0.004889146425
05items/018 | Proved   |  1315 | 1314.998748 | 0                 | 0.004864221439
05items/006 | Proved   | 1386 | 1385.901602 | 0                 | 0.004862984642
05items/038 | Proved   |  1113 | 1112.999091  | 0                 | 0.004888778552
05items/034 | Proved   | 1108 | 1107.999018  | 0                 | 0.004848460667
05items/030 | Proved   |  588 | 587.9995413 | 0                 | 0.004298817366
05items/010 | Proved   | 1952 | 1951.998264 | 0                 | 0.004889526404

rust

opened May 11, 2022 by xgillard 1

rust-polars-v0.20.0(Mar 14, 2022)

New rust polars release! :rocket:

This release of 286 commits is here thanks to the contributions of: (in no specific order):

@moritzwilksch
@JakobGM
@illumination-k
@tamasfe
@ghuls
@alexander-beedie
@Maxyme
@universalmind303
@qiemem
@glennpierce
@nmandery
@ilsley
@marcvanheerden

did I forget your contribution, please ping me, I do this manually :see_no_evil:

Most notable changes are:

Many bug fixes.
Many performance improvements.

features

Made representation of groups tuples more cache friendly #2431
Remove Seek requirement of readers
Add groupby_rolling as new entrance to expression API.
Improve CSV parsers stability and performance on several occasions
Horizontal aggregations are parallelized #2454
Reduce pivot code bloat and improve performance #2458
Struct data type added.
Extend methods that allow modification of the same memory if Arc::ref_count == 1
Avro readers and writers.
Improved rules of window expressions.
Support for us time unit.
Parquet use statistics in query optimizations.
Optimize projections in lazy computations. (Mostly useful when you deal with a large number of columns e.g. millions).
Improve performance and flexibility of melt operation @2799
new expressions
- str.split
- str.split_inclusive
- arr.join
- unique_stable
- str.split_exact
- count expression that does not require column names
- arr.arg_min
- arr.arg_max
- arr.diff
- arr.shift

Update to arrow2 0.10.0

See the 0.10.0 release for all upstream improvements.

Source code(tar.gz)
Source code(zip)

A tutorial on Julia DataFrames package

A library that wraps pandas and openpyxl and allows easy styling of dataframes in excel

Gota: DataFrames and data wrangling in Go (Golang)

An interactive grid for sorting, filtering, and editing DataFrames in Jupyter notebooks

library for working with tabular data in Julia

Blazingly fast terminal recorder that generates animated gif images for the web written in rust.

Warp is a blazingly fast, Rust-based terminal that makes you and your team more productive at running, debugging, and deploying code and infrastructure.

Memory Debugger for Windows, Linux, Mac, and Android

A Windows virtual memory editing library with support for pattern scanning.

Euboea is blazingly fast and small programming language compiled JIT to Machine code.

flume - A blazingly fast multi-producer channel

A blazingly fast online judge/ autograder ⚖️ built with Python and the Django

A blazingly fast static web server in a single binary you can set up with zero code.

🗳 minimal-feedback is a blazingly fast and highly customizable component to get user feedback.

Sonic - A blazingly fast JSON serializing & deserializing library

In-memory tabular data in Julia

A blazingly small and sane redis java client

MTuner is a C/C++ memory profiler and memory leak finder for Windows, PlayStation 4 and 3, Android and other platforms

dashmap - Blazing fast concurrent HashMap for Rust

Example of using Rust to Extend Python

Share this repo

Related Repos

Data processing

351

Powerful database anonymizer with flexible rules. Written in Rust.

Data processing

NoteCalc is a handy calculator trying to bring the advantages of Soulver to the web.

Data processing

204

amadeus - Harmonious distributed data processing & analysis in Rust

amadeus - Harmonious distributed data processing & analysis in Rust. parquet postgres aws s3 cloudfront elb json csv logs hadoop hdfs arrow common crawl

Data processing

561

WhiteboxTools is an advanced geospatial data analysis platform

WhiteboxTools is an advanced geospatial data analysis platform developed by Prof.

Data processing

1.2k

TensorBase is building a modern big data warehouse with performance in its core mind.

TensorBase is a modern engineering effort for building a high performance and cost-effective bigdata warehouse in an open source culture.

Data processing

5.7k

Blazingly fast in memory DataFrames in Rust

Polars is a DataFrames library implemented in Rust, using Apache Arrow as backend. It's focus is being a fast in memory DataFrame library that only support core functionality.

Data processing

2.8k

weld — High-performance runtime for data analytics applications

Weld Documentation Weld is a language and runtime for improving the performance of data-intensive applications. It optimizes across libraries and func

Data processing

137

utah — Dataframe structure and operations in Rust

Utah Utah is a Rust crate backed by ndarray for type-conscious, tabular data manipulation with an expressive, functional interface. Note: This crate w

Data processing

2.2k

ndarray — N-dimensional array with array views, multidimensional slicing, and efficient op...

ndarray The ndarray crate provides an n-dimensional container for general elements and for numerics. Please read the API documentation on docs.rs or t