ictonyx.data

Data handlers and loading utilities.

class ictonyx.data.DataHandler[source]

Bases: ABC

Abstract base class for all data handlers.

Provides a consistent interface regardless of data source. Handlers that load from the file system should inherit from FileDataHandler instead of this class directly.

All subclasses must implement load(), data_type, and return_format. The returned dict from load() must have keys 'train_data', 'val_data', and 'test_data', each a (X, y) tuple or None.

abstractmethod load(**kwargs)[source]

Load and split the dataset.

Returns:

Dict with keys 'train_data', 'val_data', 'test_data'.

Parameters:

kwargs (Any)

Return type:

Dict[str, Any]

abstract property data_type: str

String identifier for the data type this handler processes.

abstract property return_format: str

String describing the format returned by load().

get_data_info()[source]

Get basic information about the dataset.

Return type:

Dict[str, Any]

class ictonyx.data.ImageDataHandler(data_path, image_size, batch_size=32, seed=42, color_mode='rgb', val_split=0.2, test_split=0.1)[source]

Bases: FileDataHandler

Data handler for image classification datasets in directory format.

Expects a directory structure where each subdirectory is a class label containing image files. Images are loaded, resized, normalized to [0, 1], and split into train/val/test sets.

Requires TensorFlow for image preprocessing. Install with pip install tensorflow.

Parameters:

  • data_path (str) – Path to the root directory containing class subdirectories.

  • image_size (Tuple[int, int]) – Target (height, width) for resizing, e.g. (64, 64).

  • color_mode (str) – 'rgb' (3 channels) or 'grayscale' (1 channel). Default 'rgb'.

  • batch_size (int)

  • seed (int)

  • val_split (float)

  • test_split (float)

Raises:

ValueError – If the path is not a directory, contains no class subdirectories, or no valid image files are found.

property data_type: str

String identifier for the data type this handler processes.

property return_format: str

String describing the format returned by load().

__init__(data_path, image_size, batch_size=32, seed=42, color_mode='rgb', val_split=0.2, test_split=0.1)[source]

Initialize the image data handler.

Parameters:

  • data_path (str) – Path to directory containing class subdirectories.

  • image_size (Tuple[int, int]) – (height, width) tuple for resizing images.

  • batch_size (int) – Batch size for the datasets. Default 32.

  • seed (int) – Random seed for reproducibility. Default 42.

  • color_mode (str) – 'rgb' (default) or 'grayscale'.

  • val_split (float) – Fraction of data for validation. Default 0.2.

  • test_split (float) – Fraction of data for test. Default 0.1.

load(validation_split=0.2, test_split=0.1, **kwargs)[source]

Loads and splits the dataset into training, validation, and test sets, ensuring stratified sampling.

Parameters:

  • validation_split (float) – Proportion of data for validation

  • test_split (float) – Proportion of data for testing

  • kwargs (Any)

Returns:

Dict with ‘train_data’, ‘val_data’, ‘test_data’ as tf.data.Dataset objects

Return type:

Dict[str, Any]

get_data_info()[source]

Get comprehensive information about the image dataset.

Return type:

Dict[str, Any]

class ictonyx.data.TabularDataHandler(data=None, target_column=None, features=None, sep=',', header=0, **kwargs)[source]

Bases: FileDataHandler

Data handler for structured tabular data from CSV files or DataFrames.

Loads data, validates columns, and splits into train/val/test sets. Supports both file-path and in-memory DataFrame initialization.

Parameters:

  • data (str | DataFrame) – Path to a CSV file, or a pd.DataFrame.

  • target_column (str | None) – Name of the column containing labels.

  • features (List[str] | None) – Optional list of feature column names. If None, all columns except target_column are used.

  • sep (str) – CSV delimiter (only used for file paths). Default ','.

  • header (int) – Header row number (only used for file paths). Default 0.

Raises:

  • ValueError – If data or target_column is missing, or if the target column is not found in the data.

  • TypeError – If data is neither a string nor a DataFrame.

property data_type: str

String identifier for the data type this handler processes.

property return_format: str

String describing the format returned by load().

__init__(data=None, target_column=None, features=None, sep=',', header=0, **kwargs)[source]

Initialize the tabular data handler.

Parameters:

  • data (str | DataFrame) – Path to CSV file OR a pandas DataFrame object.

  • target_column (str | None) – Name of the target column.

  • features (List[str] | None) – List of feature column names (None = all except target).

  • sep (str) – CSV separator (only used if data is a path).

  • header (int) – Header row number (only used if data is a path).

  • **kwargs – Captures legacy ‘data_path’ argument.

load(test_split=0.2, val_split=0.1, random_state=42, **kwargs)[source]

Load and split the dataset.

Returns:

Dict with keys 'train_data', 'val_data', 'test_data'.

Parameters:
Return type:

Dict[str, Any]

get_data_info()[source]

Return metadata about the tabular dataset.

When operating from an in-memory DataFrame, returns a source key with 'dataframe' instead of a data_path / path_exists pair, which would otherwise show a nonsensical dummy path.

Return type:

Dict[str, Any]

class ictonyx.data.TextDataHandler(data_path, text_column='text', label_column='label', max_features=10000, val_split=0.1, test_split=0.2)[source]

Bases: FileDataHandler

Framework-agnostic text classification data handler.

Converts raw text to TF-IDF feature vectors compatible with all three model wrappers (Keras, PyTorch, sklearn). Does not require TensorFlow.

Requires scikit-learn: pip install ictonyx[sklearn]

Parameters:

  • data_path (str) – Path to a CSV file containing text and label columns.

  • text_column (str) – Name of the column containing raw text. Default 'text'.

  • label_column (str) – Name of the column containing labels. Default 'label'.

  • max_features (int) – Maximum number of TF-IDF features. Default 10000.

  • val_split (float) – Fraction of training data to use for validation. Default 0.1.

  • test_split (float) – Fraction of all data to hold out for testing. Default 0.2.

property data_type: str

String identifier for the data type this handler processes.

property return_format: str

String describing the format returned by load().

load(test_split=None, val_split=None, random_state=42, **kwargs)[source]

Load text data, vectorise with TF-IDF, and split.

Returns:

Dict with 'train_data', 'val_data', 'test_data' as (X, y) tuples of numpy arrays. X has shape (n_samples, max_features).

Parameters:

  • test_split (float | None)

  • val_split (float | None)

  • random_state (int)

  • kwargs (Any)

Return type:

Dict[str, Any]

get_data_info()[source]

Get information about the text dataset.

Return type:

Dict[str, Any]

class ictonyx.data.TimeSeriesDataHandler(data_path, lookback=10, sequence_length=None, value_column='value', target_column=None, stride=1, val_split=0.1, test_split=0.2)[source]

Bases: FileDataHandler

Framework-agnostic time series data handler using NumPy sliding windows.

Creates overlapping input windows from a univariate or multivariate time series. Compatible with all three model wrappers (Keras, PyTorch, sklearn). Does not require TensorFlow.

Parameters:

  • data_path (str) – Path to a CSV file containing time series data.

  • lookback (int) – Number of time steps in each input window.

  • value_column (str) – Name of the column to use as the target series. Default 'value'.

  • target_column (str | None) – Name of the column to predict. If None, uses value_column shifted by one step (next-step prediction). Default None.

  • stride (int) – Step between consecutive windows. Default 1.

  • val_split (float) – Fraction of data to use for validation (chronological). Default 0.1.

  • test_split (float) – Fraction of data to hold out for testing (chronological, taken from the end). Default 0.2.

  • sequence_length (int | None)

property data_type: str

String identifier for the data type this handler processes.

property return_format: str

String describing the format returned by load().

load(test_split=None, val_split=None, random_state=42, **kwargs)[source]

Load CSV, build sliding windows, and split chronologically.

Splits are chronological (not shuffled) to avoid data leakage. random_state is accepted for API compatibility but not used.

Returns:

Dict with 'train_data', 'val_data', 'test_data' as (X, y) tuples. X has shape (n_windows, lookback, n_features).

Parameters:

  • test_split (float | None)

  • val_split (float | None)

  • random_state (int)

  • kwargs (Any)

Return type:

Dict[str, Any]

get_data_info()[source]

Get information about the time series dataset.

Return type:

Dict[str, Any]

class ictonyx.data.ArraysDataHandler(X, y, X_test=None, y_test=None, val_split=0.1, test_split=0.2)[source]

Bases: DataHandler

Data handler for pre-loaded in-memory arrays.

Use this when data is already available as numpy arrays or Python lists, bypassing file I/O entirely. This is the handler used when passing (X, y) tuples to variability_study().

Parameters:
Raises:

ValueError – If X and y have different lengths.

property data_type: str

String identifier for the data type this handler processes.

property return_format: str

String describing the format returned by load().

__init__(X, y, X_test=None, y_test=None, val_split=0.1, test_split=0.2)[source]

Data handler for pre-loaded in-memory arrays.

Use this when data is already available as numpy arrays or Python lists, bypassing file I/O entirely. Optionally accepts a pre-held-out test set; if provided, it is passed through unchanged and only a train/val split is performed on the training arrays.

Parameters:

  • X (ndarray | List | Any) – Feature array or list. Converted to np.ndarray on init.

  • y (ndarray | List | Any) – Label array or list. Must have the same length as X.

  • X_test (ndarray | List | Any | None) – Optional pre-held-out test features. If provided, load() will not carve a test set from X/y.

  • y_test (ndarray | List | Any | None) – Optional pre-held-out test labels. Required if X_test is provided.

  • val_split (float) – Default validation fraction passed to load()

  • 0.1. (when not overridden there. Default)

  • test_split (float) – Default test fraction passed to load()

  • 0.2. (when not overridden there. Default)

Raises:

  • ValueError – If X and y have different lengths.

  • ValueError – If exactly one of X_test / y_test is provided.

  • ValueError – If X_test and y_test have different lengths.

load(test_split=None, val_split=None, random_state=42, **kwargs)[source]

Split the pre-loaded arrays.

If X_test and y_test were provided at construction, they are used as the test set directly and only a train/val split is performed. Otherwise, test_split is carved out of the provided arrays first.

Parameters:

  • test_split (float | None) – Fraction to reserve for test. Defaults to the value passed to __init__ (default 0.2). Pass 0 to skip the test split.

  • val_split (float | None) – Fraction to reserve for validation. Defaults to the value passed to __init__ (default 0.1).

  • random_state (int) – Seed for reproducibility. Default 42.

  • kwargs (Any)

Return type:

Dict[str, Any]

ictonyx.data.auto_resolve_handler(data, target_column=None, **kwargs)[source]

Factory function to automatically detect and initialize the appropriate DataHandler.

Parameters:

  • data (str | DataFrame | Tuple[ndarray, ndarray] | Any) – The input data. Supported formats: - str: Path to a directory (Images) - str: Path to a CSV file (Tabular, Text, or TimeSeries) - pd.DataFrame: Data object (Tabular) - tuple: (X, y) arrays (Arrays)

  • target_column (str | None) – Name of target column (Required for DataFrame/Tabular).

  • **kwargs – Additional arguments passed to the specific handler constructor (e.g., ‘image_size’, ‘text_column’, ‘sequence_length’).

Returns:

An initialized instance of the appropriate handler.

Return type:

DataHandler

Raises: