brainbox.io.parquet

Functions

load

Loads parquet file into pandas dataframe :param file: :return:

np2str

np2uuid

rec2col

Change a record list (usually from a REST API endpoint) to a column based dictionary (pandas dataframe).

save

Save pandas dataframe to parquet :param file: :param table: :return:

str2np

Converts uuid string or list of uuid strings to int64 numpy array with 2 cols Returns [0, 0] for None list entries

uuid2np

load(file)[source]

Loads parquet file into pandas dataframe :param file: :return:

save(file, table)[source]

Save pandas dataframe to parquet :param file: :param table: :return:

uuid2np(eids_uuid)[source]
str2np(eids_str)[source]

Converts uuid string or list of uuid strings to int64 numpy array with 2 cols Returns [0, 0] for None list entries

np2uuid(eids_np)[source]
np2str(eids_np)[source]
rec2col(rec, join=None, include=None, exclude=None, uuid_fields=None, types=None)[source]

Change a record list (usually from a REST API endpoint) to a column based dictionary (pandas dataframe). :param rec (list): list of dictionaries with consistent keys :param join (dictionary): dictionary of scalar keys that will be replicated over the full array (join operation) :param include: list of strings representing dictionary keys: if specified will only include the keys specified here :param exclude: list of strings representing dictionary keys: if specified will exclude the keys specified here :param uuid_fields: if the field is an UUID, will split it into 2 distinct int64 columns for efficient lookups and intersections :param types: for a given key, will force the type; example: types = {‘file_size’: np.double} :return: a Bunch