# Load\_dataset\_builder verification

**URL:** <https://discuss.huggingface.co/t/load-dataset-builder-verification/67114>\
**Category:** 🤗Datasets\
**Created:** [December 27, 2023, 4:54am UTC](https://discuss.huggingface.co/t/load-dataset-builder-verification/67114 "2023-12-27T04:54:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Owos](https://avatars.discourse-cdn.com/v4/letter/o/87869e/32.png) [@Owos](https://discuss.huggingface.co/u/Owos)\
**Post date:** [December 27, 2023, 4:54am UTC](https://discuss.huggingface.co/t/load-dataset-builder-verification/67114/1 "2023-12-27T04:54:33Z")

</div>

I am trying to load a subset of a dataset from the hub using `load_dataset_builder` but the verification keeps throwing an error. I have edited the `ReadMe` to reflect the number that I want which is `2201939` but each time I run the download script, it overrides the Readme file and changes the number.

`datasets.utils.info_utils.NonMatchingSplitsSizesError: [{'expected': SplitInfo(name='train', num_bytes=14752554924, num_examples=40836715, shard_lengths=None, dataset_name=None), 'recorded': SplitInfo(name='train', num_bytes=724931127, num_examples=2201939, shard_lengths=[1516000, 685939], dataset_name='wmt_utils')}]`

code that produces the error:

```auto
inspect_dataset("wmt14", "data/path2/to/scripts2")
    builder = load_dataset_builder(
        "data/path2/to/scripts2/wmt_utils.py",
        language_pair=("fr", "en"),
        subsets={
            datasets.Split.TRAIN: ["europarl_v7", "newscommentary_v10"],
            datasets.Split.VALIDATION: ["newstest2013"],
            datasets.Split.TEST: ["newstest2014"],

```

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [January 12, 2024, 2:49pm UTC](https://discuss.huggingface.co/t/load-dataset-builder-verification/67114/2 "2024-01-12T14:49:36Z")

</div>

Hi ! This looks like a bug 😕  
Maybe you can disable verifications using `verification_mode="no_checks"` ?

---

<div class="post-metadata">

**Author:** ![Owos](https://avatars.discourse-cdn.com/v4/letter/o/87869e/32.png) [@Owos](https://discuss.huggingface.co/u/Owos)\
**Post date:** [January 12, 2024, 7:25pm UTC](https://discuss.huggingface.co/t/load-dataset-builder-verification/67114/3 "2024-01-12T19:25:50Z")

</div>

I was the one making a mistake, commenting out the first line solves the problem.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [February 5, 2024, 4:37am UTC](https://discuss.huggingface.co/t/load-dataset-builder-verification/67114/4 "2024-02-05T04:37:15Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.