polymatheia.data.writer
This module provides a writer for serialising data to the local filesystem.
- class polymatheia.data.writer.CSVWriter(target, default_value='', extras_action='ignore', column_names=None)
The
CSVWriter
writes records into a CSV file.The
CSVWriter
assumes that no record contains any kind of nested data. If it is passed nested data, then the behaviour is undefined.- __init__(target, default_value='', extras_action='ignore', column_names=None)
Create a new
CSVWriter
.- Parameters:
target – The target to write the CSV to. Can either be a
str
filename or an existing file-like objectdefault_value – The default value to output if a record does not contain a value for a specified CSV column name
extras_action (
str
) – The action to take if a record contains keys that are not in the CVS fieldnames. Set to'ignore'
to just ignore this (the default). Set to'raise'
to raise aValueError
.fieldnames (
list
ofstr
) – The CSV column names to use. IfNone
is specified, then the column names are derived from the first record’s keys.
- __weakref__
list of weak references to the object (if defined)
- write(records)
Write the
records
to the CSV file.- Parameters:
records (Iterable of
NavigableDict
) – The records to write
- class polymatheia.data.writer.JSONWriter(directory, id_path)
The
JSONWriter
writes records to the local filesystem as JSON files.- __init__(directory, id_path)
Create a new
JSONWriter
.For each record the identifier is used to create a directory structure. In the leaf directory the identifier is then used as the filename.
- Parameters:
directory (
str
) – The base directory within which to create the filesid_path (
str
orlist
) – The path used to access the identifier in the record
- __weakref__
list of weak references to the object (if defined)
- write(records)
Write the records to the file-system.
- Parameters:
records (Iterable of
NavigableDict
) – The records to write
- class polymatheia.data.writer.PandasDFWriter
The
PandasDFWriter
writes records to a PandasDataFrame
.The
PandasDFWriter
attempts to automatically coerce columns to integers or floats.The
PandasDFWriter
assumes that no record contains any kind of nested data. If it is passed nested data, then the behaviour is undefined.- __init__()
Create a new
PandasDFWriter
.
- __weakref__
list of weak references to the object (if defined)
- write(records)
Write the
records
to the PandasDataFrame
.- Parameters:
records (Iterable of
NavigableDict
) – The records to write- Returns:
The Pandas dataframe
- Return type:
- class polymatheia.data.writer.XMLWriter(directory, id_path)
The
XMLWriter
writes records to the local filesystem as XML.- __init__(directory, id_path)
Create a new
XMLWriter
.For each record the identifier is used to create a directory structure. In the leaf directory the identifier is then used as the filename.
- Parameters:
directory (
str
) – The base directory within which to create the filesid_path (
str
orlist
) – The path used to access the identifier in the record
- __weakref__
list of weak references to the object (if defined)
- write(records)
Write the records to the file-system.
- Parameters:
records (Iterable of
NavigableDict
) – The records to write