brainbox.io.parquet

Functions

load

Loads parquet file into pandas dataframe

np2str

np2uuid

rec2col

Change a record list (usually from a REST API endpoint) to a column based dictionary (pandas dataframe).

save

Save pandas dataframe to parquet

str2np

Converts uuid string or list of uuid strings to int64 numpy array with 2 cols Returns [0, 0] for None list entries

uuid2np

load(file)[source]

Loads parquet file into pandas dataframe

Parameters

file

Returns

save(file, table)[source]

Save pandas dataframe to parquet

Parameters
  • file

  • table

Returns

uuid2np(eids_uuid)[source]
str2np(eids_str)[source]

Converts uuid string or list of uuid strings to int64 numpy array with 2 cols Returns [0, 0] for None list entries

np2uuid(eids_np)[source]
np2str(eids_np)[source]
rec2col(rec, join=None, include=None, exclude=None, uuid_fields=None, types=None)[source]

Change a record list (usually from a REST API endpoint) to a column based dictionary (pandas dataframe).

Parameters
  • (list) (rec) – list of dictionaries with consistent keys

  • (dictionary) (join) – dictionary of scalar keys that will be replicated over the full

array (join operation) :param include: list of strings representing dictionary keys: if specified will only include the keys specified here :param exclude: list of strings representing dictionary keys: if specified will exclude the keys specified here :param uuid_fields: if the field is an UUID, will split it into 2 distinct int64 columns for efficient lookups and intersections :param types: for a given key, will force the type; example: types = {‘file_size’: np.double} :return: a Bunch