Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement DynamoDBWrite class. #460

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

nasudadada
Copy link
Contributor

Summary

Add functionality to write data from CSV or JSONL files to DynamoDB tables

Key features

  • Specify input file path, table name, and file format
  • Support for CSV and JSONL input formats

This commit enables users to efficiently import data into DynamoDB tables from CSV or JSONL files.

Brief

This PR adds a new feature to write data from CSV or JSONL files to DynamoDB tables. Key features include:

User-specifiable parameters:

  • Input file path
  • Table name
  • File format (CSV or JSONL)

Supported input formats:

  • CSV
  • JSONL

The implementation uses batch write operations to efficiently insert data into the DynamoDB table. This feature addition enables users to import datasets into DynamoDB tables from common file formats.

Points to Check

  • Is the current approach to exception handling and logging consistent with other parts of the project? Should we enhance error reporting?
  • Are there any implementation aspects missing from this OSS implementation? Please check if I've overlooked any considerations, particularly regarding data consistency or performance during writes.
  • Does this implementation align with the style of this OSS project?

Test

Confirming

Review Limit

  • None

Additional Information:
In the current implementation, we're focusing on basic write operations. For future enhancements, we could consider:

  • Adding support for updating existing items in the table
  • Implementing a feature to map CSV/JSONL fields to specific DynamoDB attributes
  • Adding options for handling duplicate keys or other write conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant