Learning and Thoughts on Object Storage

sources:

• https://vutr.substack.com/p/amazon-s3-for-data-engineers?utm_source=publication-search

For each object in a bucket, it has:

• key -> the unique key, which serves as its identifier
• prefix -> used for pretending in an organised folder structure
  • used in partitioning objects for load balancing
  • used in searching and returning objects with a same prefix
• version ID -> key for multiple versions of the same object
• value -> actual content stored, supporting any data in any format
  • viewed as a sequence of bytes
• metadata
  • system metadata -> assigned by S3 to manage the object
  • user defined metadata -> assigned by users, e.g. tags