ertk.preprocessing.huggingface.HuggingFaceExtractorConfig

class ertk.preprocessing.huggingface.HuggingFaceExtractorConfig(model: str = '???', device: str = 'cuda', task: Task = Task.CTC, agg: Agg | None = Agg.MEAN, layer: str | None = 'context', max_input_len: int = 1500000, whisper_lang: str | None = None, max_new_tokens: int = 448)

Bases: ERTKConfig

HuggingFace feature extractor configuration.

__init__(model: str = '???', device: str = 'cuda', task: Task = Task.CTC, agg: Agg | None = Agg.MEAN, layer: str | None = 'context', max_input_len: int = 1500000, whisper_lang: str | None = None, max_new_tokens: int = 448) → None

Methods

__init__([model, device, task, agg, layer, ...])

Inherited Methods

default()	Create default config.
from_config(config)	Create config object from any compatible config.
from_file(path)	Create config from YAML file and optionlly override some values.
merge_with_args(args)	Merge config with command-line arguments.
merge_with_config(config)	Merge other config into this config.
to_dictconfig()	Convert config to DictConfig.
to_file(path)	Write config to YAML file.
to_string()	Generate YAML string representation of config.

Attributes

agg	Aggregation method for embeddings.
device	Device to run model on.
layer	Layer to extract embeddings from.
max_input_len	Maximum input length.
max_new_tokens	Maximum number of generated tokens for speech2text and Whisper models.
model	Model name or path.
task	Task to perform.
whisper_lang	Language to use for Whisper models.

agg: Agg | None = 'mean'

Aggregation method for embeddings.

device: str = 'cuda'

Device to run model on.

layer: str | None = 'context'

Layer to extract embeddings from.

max_input_len: int = 1500000

Maximum input length.

max_new_tokens: int = 448

Maximum number of generated tokens for speech2text and Whisper models.

model: str = '???'

Model name or path.

task: Task = 'ctc'

Task to perform.

whisper_lang: str | None = None

Language to use for Whisper models.