From c6bf6c3068812c5d01c50e263d4b57bfcbe74a1b Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Thu, 20 Mar 2025 16:37:07 -0400 Subject: [PATCH 1/2] Remove object_store references in readme.md --- README.md | 32 +++++++------------------------- 1 file changed, 7 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 89b808785aed..1513f9542cad 100644 --- a/README.md +++ b/README.md @@ -21,29 +21,31 @@ Welcome to the [Rust][rust] implementation of [Apache Arrow], the popular in-memory columnar format. -This repo contains the following main components: +This repository contains the following crates: | Crate | Description | Latest API Docs | README | | ------------------ | ---------------------------------------------------------------------------- | ------------------------------------------------ | --------------------------------- | | [`arrow`] | Core functionality (memory layout, arrays, low level computations) | [docs.rs](https://docs.rs/arrow/latest) | [(README)][arrow-readme] | | [`arrow-flight`] | Support for Arrow-Flight IPC protocol | [docs.rs](https://docs.rs/arrow-flight/latest) | [(README)][flight-readme] | -| [`object-store`] | Support for object store interactions (aws, azure, gcp, local, in-memory) | [docs.rs](https://docs.rs/object_store/latest) | [(README)][objectstore-readme] | | [`parquet`] | Support for Parquet columnar file format | [docs.rs](https://docs.rs/parquet/latest) | [(README)][parquet-readme] | | [`parquet_derive`] | A crate for deriving RecordWriter/RecordReader for arbitrary, simple structs | [docs.rs](https://docs.rs/parquet-derive/latest) | [(README)][parquet-derive-readme] | The current development version the API documentation in this repo can be found [here](https://arrow.apache.org/rust). +Note: previously the [`object-store`] crate was also part of this repository, +but it has been moved to the [arrow-rs-object-store repository] + + [apache arrow]: https://arrow.apache.org/ [`arrow`]: https://crates.io/crates/arrow [`parquet`]: https://crates.io/crates/parquet [`parquet_derive`]: https://crates.io/crates/parquet-derive [`arrow-flight`]: https://crates.io/crates/arrow-flight [`object-store`]: https://crates.io/crates/object-store +[arrow-rs-object-store repository]: https://github.com/apache/arrow-rs-object-store ## Release Versioning and Schedule -### `arrow` and `parquet` crates - The Arrow Rust project releases approximately monthly and follows [Semantic Versioning]. @@ -72,21 +74,6 @@ Planned Release Schedule [ticket #5368]: https://github.com/apache/arrow-rs/issues/5368 [semantic versioning]: https://semver.org/ -### `object_store` crate - -The [`object_store`] crate is released independently of the `arrow` and -`parquet` crates and follows [Semantic Versioning]. We aim to release new -versions approximately every 2 months. - -[`object_store`]: https://crates.io/crates/object_store - -Planned Release Schedule - -| Approximate Date | Version | Notes | -| ---------------- | -------- | --------------------------------------- | -| Feb 2025 | `0.12.0` | Major, potentially breaking API changes | -| Apr 2025 | `0.12.1` | Minor, NO breaking API changes | - ### Guidelines for `panic` vs `Result` In general, use panics for bad states that are unreachable, unrecoverable or harmful. @@ -132,14 +119,10 @@ There are several related crates in different repositories | ------------------------ | ------------------------------------------- | --------------------------------------- | | [`datafusion`] | In-memory query engine with SQL support | [(README)][datafusion-readme] | | [`ballista`] | Distributed query execution | [(README)][ballista-readme] | -| [`object_store_opendal`] | Use [`opendal`] as [`object_store`] backend | [(README)][object_store_opendal-readme] | | [`parquet_opendal`] | Use [`opendal`] for [`parquet`] Arrow IO | [(README)][parquet_opendal-readme] | [`datafusion`]: https://crates.io/crates/datafusion [`ballista`]: https://crates.io/crates/ballista -[`object_store_opendal`]: https://crates.io/crates/object_store_opendal -[`opendal`]: https://crates.io/crates/opendal -[object_store_opendal-readme]: https://github.com/apache/opendal/blob/main/integrations/object_store/README.md [`parquet_opendal`]: https://crates.io/crates/parquet_opendal [parquet_opendal-readme]: https://github.com/apache/opendal/blob/main/integrations/parquet/README.md @@ -170,7 +153,7 @@ a great place to meet other contributors and get guidance on where to contribute The Rust implementation uses [GitHub issues][issues] as the system of record for new features and bug fixes and this plays a critical role in the release process. -For design discussions we generally collaborate on Google documents and file a GitHub issue linking to the document. +For design discussions we generally use GitHub issues. There is more information in the [contributing] guide. @@ -181,7 +164,6 @@ There is more information in the [contributing] guide. [flight-readme]: arrow-flight/README.md [datafusion-readme]: https://github.com/apache/datafusion/blob/main/README.md [ballista-readme]: https://github.com/apache/datafusion-ballista/blob/main/README.md -[objectstore-readme]: object_store/README.md [parquet-derive-readme]: parquet_derive/README.md [issues]: https://github.com/apache/arrow-rs/issues [discussions]: https://github.com/apache/arrow-rs/discussions From dfcaa64ba22d5da194dec4fc1ed7aefbb188a210 Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Fri, 21 Mar 2025 12:32:24 -0400 Subject: [PATCH 2/2] Add object_store as a related crate --- README.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 1513f9542cad..6ac4c838d655 100644 --- a/README.md +++ b/README.md @@ -32,16 +32,14 @@ This repository contains the following crates: The current development version the API documentation in this repo can be found [here](https://arrow.apache.org/rust). -Note: previously the [`object-store`] crate was also part of this repository, +Note: previously the [`object_store`] crate was also part of this repository, but it has been moved to the [arrow-rs-object-store repository] - [apache arrow]: https://arrow.apache.org/ [`arrow`]: https://crates.io/crates/arrow [`parquet`]: https://crates.io/crates/parquet [`parquet_derive`]: https://crates.io/crates/parquet-derive [`arrow-flight`]: https://crates.io/crates/arrow-flight -[`object-store`]: https://crates.io/crates/object-store [arrow-rs-object-store repository]: https://github.com/apache/arrow-rs-object-store ## Release Versioning and Schedule @@ -115,16 +113,18 @@ maintainers. There are several related crates in different repositories -| Crate | Description | Documentation | -| ------------------------ | ------------------------------------------- | --------------------------------------- | -| [`datafusion`] | In-memory query engine with SQL support | [(README)][datafusion-readme] | -| [`ballista`] | Distributed query execution | [(README)][ballista-readme] | -| [`parquet_opendal`] | Use [`opendal`] for [`parquet`] Arrow IO | [(README)][parquet_opendal-readme] | +| Crate | Description | Documentation | +| ------------------- | ------------------------------------------------------------ | ---------------------------------- | +| [`object_store`] | Object Storage (aws, azure, gcp, local, in-memory) interface | [(README)](object_store-readme) | +| [`datafusion`] | In-memory query engine with SQL support | [(README)][datafusion-readme] | +| [`ballista`] | Distributed query execution | [(README)][ballista-readme] | +| [`parquet_opendal`] | Use [`opendal`] for [`parquet`] Arrow IO | [(README)][parquet_opendal-readme] | [`datafusion`]: https://crates.io/crates/datafusion [`ballista`]: https://crates.io/crates/ballista [`parquet_opendal`]: https://crates.io/crates/parquet_opendal [parquet_opendal-readme]: https://github.com/apache/opendal/blob/main/integrations/parquet/README.md +[object_store-readme]: https://github.com/apache/arrow-rs-object-store/blob/main/README.md Collectively, these crates support a wider array of functionality for analytic computations in Rust. @@ -158,6 +158,7 @@ For design discussions we generally use GitHub issues. There is more information in the [contributing] guide. [rust]: https://www.rust-lang.org/ +[`object_store`]: https://crates.io/crates/object-store [arrow-readme]: arrow/README.md [contributing]: CONTRIBUTING.md [parquet-readme]: parquet/README.md