Hugging Face Datasets #3 | Adding Images

Описание

How to work with the Hugging Face datasets library in Python. Here we focus on adding images, using dataset builder scripts, the download manager and iter_archive function. Everything we do is using the best-practice methods for Hugging Face (huggingface) Datasets, all in Python.

Can be used for datasets in image search, similarity search/semantic search/vector similarity search, classification, question-answering. Makes training/fine-tuning models with pytorch and tensorflow easy.

? AI Dev Studio:
https://aurelio.ai/

? Discord:
https://discord.gg/c5QtDB9RAP

00:00 Intro
02:05 Creating Tar Files for Images
05:11 Compressing Images in Tar Files
06:26 Adding Dataset Builder Script
09:07 Iterable Download Manager with iter_archive
09:56 _generate_examples Function Definition
12:52 Adding to Hugging Face Datasets Hub
13:34 Fixing Errors
14:23 Using Your New Dataset
14:53 Dealing with Larger Image Datasets