Skip to content

Dataset

Bases: BaseModel

A dataset class representing a dataset in the Tenyks platform

Attributes:

Name Type Description
client Client

The client to interact with the Tenyks API.

workspace_name str

Name of the workspace the dataset belongs to.

key str

Key of the dataset.

name str

Name of the dataset.

owner str

Owner of the dataset.

owner_email EmailStr

Owner email of the dataset.

created_at datetime

Creation timestamp of the dataset.

images_location Optional[Union[AWSLocation, AzureLocation, GCSLocation]]

Directory location of the images of the dataset.

metadata_location Optional[Union[AWSLocation, AzureLocation, GCSLocation]]

Directory location of the metadata of the dataset.

categories List[Category]

Categories/classes of the dataset.

models List

Names of the models of the dataset.

status str

Status of the dataset.

n_images int

Number of images in the dataset.

iou_threshold float

IOU threshold set for the dataset.

add_image(image_path, annotations=None, tags=None, verbose=False)

Add an image to the dataset along with its annotations and tags.

Parameters:

Name Type Description Default
image_path str

The path of the image to add.

required
annotations Optional[List[Annotation]]

The annotations to add to the image. Defaults to None.

None
tags Optional[List[Tag]]

The tags to add to the image. Defaults to None.

None
verbose Optional[bool]

If True, provides progress updates. Defaults to False.

False

count_images(filter=None, model_key=None)

Return image count that match the filter criteria.

Parameters:

Name Type Description Default
filter Optional[str]

Filter conditions for counting. Defaults to None.

None
model_key Optional[str]

Model key to filter images. Defaults to None.

None

Returns:

Name Type Description
int int

Number of images that match the filter criteria.

create_model(name, confidence_threshold=None, iou_threshold=None)

Create a new model for the dataset.

Parameters:

Name Type Description Default
name str

The name of the new model.

required
confidence_threshold Optional[float]

The confidence threshold for the model. Defaults to None.

None
iou_threshold Optional[float]

The IOU threshold for the model. Defaults to None.

None

Returns:

Name Type Description
Model Model

The newly created model.

delete_model(key)

Delete a model from the dataset.

Parameters:

Name Type Description Default
key str

The key of the model to delete.

required

finetune_search_model(search_query, ground_truth_search_results)

Placeholder method for finetuning search

Parameters:

Name Type Description Default
search_query str

search query on which to finetune model

required
ground_truth_search_results List[Image]

ground truth images that should be retrieved

required

get_category_by_id(category_id)

Retrieve a category by its ID.

Parameters:

Name Type Description Default
category_id int

The ID of the category to retrieve.

required

Returns:

Name Type Description
Category Category

The category corresponding to the given ID.

get_category_by_name(category_name)

Retrieve a category by its name.

Parameters:

Name Type Description Default
category_name str

The name of the category to retrieve.

required

Returns:

Name Type Description
Category Category

The category corresponding to the given name.

get_image_by_key(image_key)

Retrieve an image by its key.

Parameters:

Name Type Description Default
image_key str

The key of the image to retrieve.

required

Returns:

Name Type Description
Image Image

The image corresponding to the given key.

get_model(key)

Retrieve a model by its key.

Parameters:

Name Type Description Default
key str

The key of the model to retrieve.

required

Returns:

Name Type Description
Model Model

The model corresponding to the given key.

get_model_names()

Retrieve the names of the models associated with the dataset.

Returns:

Type Description
List[str]

List[str]: A list of model display names.

get_models()

Retrieve the models associated with the dataset.

Returns:

Type Description
List[Model]

List[Model]: A list of models associated with the dataset.

get_tag_by_key(tag_key)

Retrieve a tag by its key.

Parameters:

Name Type Description Default
tag_key str

The key of the tag to retrieve.

required

Returns:

Name Type Description
Tag Tag

The tag corresponding to the given key.

get_tag_by_name(tag_name)

Retrieve a tag by its display name.

Parameters:

Name Type Description Default
tag_name str

The name of the tag to retrieve.

required

Returns:

Name Type Description
Tag Tag

The tag corresponding to the given display name.

get_tags()

Retrieve the tags associated with the dataset.

Returns:

Type Description
List[Tag]

List[Tag]: A list of tags created for the dataset.

head(n=5)

Retrieve the first few images from the dataset.

Parameters:

Name Type Description Default
n int

The number of images to retrieve. Defaults to 5.

5

Returns:

Type Description
List[Image]

List[Image]: A list of the first n images in the dataset.

images_generator(filter=None, sort_by=None, model_key=None, page_size=250)

Generator to retrieve images from the dataset in a paginated manner.

Parameters:

Name Type Description Default
filter Optional[str]

Filter conditions for the search. Defaults to None.

None
sort_by Optional[str]

Sort criteria for the search. Defaults to None.

None
model_key Optional[str]

Model key to filter images. Defaults to None.

None
page_size Optional[int]

Number of images per page. Defaults to 250.

250

Yields:

Name Type Description
Generator Generator

A generator yielding images.

ingest(import_operation=None, verbose=True)

Trigger the ingestion process for the dataset.

Parameters:

Name Type Description Default
import_operation Optional[str]

The import operation type. Defaults to None.

None
verbose Optional[bool]

If True, provides progress updates. Defaults to True.

True

save_image_metadata(metadata_key, metadata_values)

Add or update custom metadata for images in a dataset.

Parameters:

Name Type Description Default
metadata_key str

The key representing the type of metadata to be saved. Must contain only alphanumeric characters (no spaces, underscores, or special characters), e.g. brightness.

required
metadata_values Dict[str, Union[int, float]]

A dictionary where the keys are image identifiers and the values are the metadata values to be saved (either integer or float).

required
Example

metadata_values = { "image1": 0.75, "image2": 0.85, "image3": 0.65, # More image metadata... } dataset.save_image_metadata( metadata_key="brightness", metadata_values=metadata_values )

Note

The metadata values are sent to the server in batches of 500 to avoid overwhelming the API. Each batch is processed sequentially, and the method logs the progress of each batch. After all batches are processed, the dataset's metadata key is updated accordingly.

search_images(n_images=250, filter=None, sort_by=None, model_key=None)

Perform image search in the dataset based on filters.

Parameters:

Name Type Description Default
n_images Optional[int]

The number of images to retrieve. Defaults to 250.

250
filter Optional[str]

Filter conditions for the search. Defaults to None.

None
sort_by Optional[str]

Sort criteria for the search. Defaults to None.

None
model_key Optional[str]

Model key to filter images. Defaults to None.

None

Returns:

Type Description
List[Image]

List[Image]: A list of images that match the search criteria.

search_video(n_videos=50, filter=None, sort_by=None, model_key=None)

Perform video search in the dataset based on filters.

Parameters:

Name Type Description Default
n_videos Optional[str]

Number of video clips to return. Defaults to 50.

50
filter Optional[str]

Filter conditions for the search. Defaults to None.

None
sort_by Optional[str]

Sort criteria for the search. Defaults to None.

None
model_key Optional[str]

Model key to filter videos. Defaults to None.

None

Returns:

Type Description
List[VideoClip]

List[VideoClip]: A list of video clips that match the search criteria.

update_image(image_key, annotations, tags=None, verbose=False)

Update an existing image's annotations and tags.

Parameters:

Name Type Description Default
image_key str

The key of the image to update.

required
annotations List[Annotation]

The new annotations for the image.

required
tags Optional[List[Tag]]

The new tags for the image. Defaults to None.

None
verbose Optional[bool]

If True, provides progress updates. Defaults to False.

False

upload_annotations(coco_path_or_dict, verbose=True)

Upload annotations to the dataset.

Parameters:

Name Type Description Default
coco_path_or_dict Union[str, dict]

The file path or dictionary of COCO annotations to upload.

required
verbose Optional[bool]

If True, provides progress updates. Defaults to True.

True

upload_annotations_from_cloud(coco_file_location)

Upload annotations to the dataset from a cloud location.

Parameters:

Name Type Description Default
coco_file_location Union[AWSLocation, AzureLocation, GCSLocation]

The cloud location of the COCO annotations to upload.

required

upload_custom_embeddings(embedding_name, embedding_location, embedding_type='images', verbose=True)

Upload custom embeddings to the dataset for use in Embedding viewer.

Parameters:

Name Type Description Default
embedding_name str

The display name of the embeddings.

required
embedding_location dict

The location of the embeddings in cloud storage.

required
embedding_type str

The type of embeddings. At present only 'images' is supported. 'annotations'/'predictions' coming soon!

'images'
verbose Optional[bool]

If True, provides progress updates. Defaults to True.

True

upload_custom_embeddings_from_local(embedding_name, embedding_path, embedding_type='images', verbose=True)

Upload custom embeddings from a local file to the dataset.

Parameters:

Name Type Description Default
embedding_name str

The display name of the embeddings.

required
embedding_path str

The path to the custom embeddings JSON.

required
embedding_type str

The type of embeddings. At present only 'images' is supported. 'annotations'/'predictions' coming soon!

'images'
verbose Optional[bool]

If True, provides progress updates. Defaults to True.

True

upload_images(image_directory_or_paths, verbose=True)

Upload images to the dataset.

Parameters:

Name Type Description Default
image_directory_or_paths Union[str, Path, List[str]]

The directory or paths of the images to upload.

required
verbose Optional[bool]

If True, provides progress updates. Defaults to True.

True

upload_videos_from_cloud_and_ingest(video_folder_location, sample_rate_per_second, frames_to_subsample, prompts=['objects'], threshold=0.005)

Create a new dataset in the current workspace.

Parameters:

Name Type Description Default
video_folder_location Union[AWSLocation, GCSLocation, AzureLocation]

The location of the folder of videos where the images uploaded to the dataset come from

required

video_clip_generator(filter=None, sort_by=None, model_key=None, page_size=50)

Generator to retrieve video clips from the dataset in a paginated manner.

Parameters:

Name Type Description Default
filter Optional[str]

Filter conditions for the search. Defaults to None.

None
sort_by Optional[str]

Sort criteria for the search. Defaults to None.

None
model_key Optional[str]

Model key to filter videos. Defaults to None.

None
page_size Optional[int]

Number of images per page. Defaults to 50.

50

Yields:

Name Type Description
Generator Generator

A generator yielding images.