Skip to content

Data Sync

Data synchronization models and utilities.


BatchDataSyncRequest

Bases: PydanticBaseModel

Request for a batch of data sync operations.

BatchDataSyncResponse

Bases: PydanticBaseModel

Response from a batch data sync operation.

add_failed_request

add_failed_request(request)

Add a request to the list of failed requests.

Source code in src/aibs_informatics_core/models/data_sync.py
186
187
188
189
190
def add_failed_request(self, request: DataSyncRequest) -> None:
    """Add a request to the list of failed requests."""
    if self.failed_requests is None:
        self.failed_requests = []
    self.failed_requests.append(request)

BatchDataSyncResult

Bases: DataSyncResult

Aggregated result metrics for a batch data sync.

increment_failed_requests_count

increment_failed_requests_count(increment=1)

Increment the failed and total request counters.

Source code in src/aibs_informatics_core/models/data_sync.py
174
175
176
177
def increment_failed_requests_count(self, increment: int = 1) -> None:
    """Increment the failed and total request counters."""
    self.failed_requests_count += increment
    self.total_requests_count += increment

increment_successful_requests_count

increment_successful_requests_count(increment=1)

Increment the successful and total request counters.

Source code in src/aibs_informatics_core/models/data_sync.py
169
170
171
172
def increment_successful_requests_count(self, increment: int = 1) -> None:
    """Increment the successful and total request counters."""
    self.successful_requests_count += increment
    self.total_requests_count += increment

DataSyncConfig

Bases: PydanticBaseModel

Configuration options for data sync operations.

DataSyncRequest

Bases: DataSyncConfig, DataSyncTask

Combined request model for a single data sync operation.

config property

config

Extract the configuration portion of this request.

task property

task

Extract the task portion of this request.

DataSyncResponse

Bases: PydanticBaseModel

Response from a single data sync operation.

DataSyncResult

Bases: PydanticBaseModel

Result metrics for a data sync operation.

add_bytes_transferred

add_bytes_transferred(bytes_transferred)

Increment the bytes transferred counter.

Source code in src/aibs_informatics_core/models/data_sync.py
128
129
130
def add_bytes_transferred(self, bytes_transferred: int) -> None:
    """Increment the bytes transferred counter."""
    self.bytes_transferred += bytes_transferred

add_files_transferred

add_files_transferred(files_transferred)

Increment the files transferred counter.

Source code in src/aibs_informatics_core/models/data_sync.py
132
133
134
def add_files_transferred(self, files_transferred: int) -> None:
    """Increment the files transferred counter."""
    self.files_transferred += files_transferred

DataSyncTask

Bases: PydanticBaseModel

Defines source and destination paths for a data sync operation.

GetJSONFromFileRequest

Bases: JSONReference

Request to read JSON content from a file.

GetJSONFromFileResponse

Bases: JSONContent

Response containing JSON content read from a file.

JSONContent

Bases: PydanticBaseModel

Model containing raw JSON content.

JSONReference

Bases: PydanticBaseModel

Model containing a reference to a JSON file.

PrepareBatchDataSyncRequest

Bases: DataSyncRequest

Request to prepare a batch of data sync operations from a single sync task.

PrepareBatchDataSyncResponse

Bases: PydanticBaseModel

Response containing prepared batch data sync requests.

PutJSONToFileRequest

Bases: JSONContent

Request to write JSON content to a file.

PutJSONToFileResponse

Bases: JSONReference

Response from writing JSON content to a file.

RemoteToLocalConfig

Bases: PydanticBaseModel

Configuration for syncing remote data to local filesystem.