fenn.datasets

class fenn.datasets.TextDataset(X, y, tokenizer, max_length=1024)

Bases: Dataset[tuple[Tensor, Tensor] | tuple[Tensor, Tensor, Tensor]]

Generic text + binary label dataset. X: list[str] y: list[int|float]

Parameters:

• X (Sequence[str])

• y (Sequence[int | float] | None)

• tokenizer (TextTokenizer)

• max_length (int)

__init__(X, y, tokenizer, max_length=1024)

Parameters:

• X (Sequence[str])

• y (Sequence[int | float] | None)

• tokenizer (TextTokenizer)

• max_length (int)

Return type:

None