TypeError: TextInputSequence must be str – Hugging Face Transformers squad_convert_examples_from_dataset Example usage

by

Example usage is

https://huggingface.co/docs/transformers/v4.32.0/en/main_classes/processors#example-usage

# tensorflow_datasets only handle Squad V1.
tfds_examples = tfds.load("squad")
examples = SquadV1Processor().get_examples_from_dataset(tfds_examples, evaluate=evaluate)

features = squad_convert_examples_to_features(
    examples=examples,
    tokenizer=tokenizer,
    max_seq_length=max_seq_length,
    doc_stride=args.doc_stride,
    max_query_length=max_query_length,
    is_training=not evaluate,
)

When I run this example-usage code, I get the error: TypeError: TextInputSequence must be str.

The tokenizer I used is:

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

thank you.

Leave a Comment