feat: add SchemaProvider::table_type(table_name: &str)

[epgif]

InformationSchemaConfig::make_tables only needs the TableType not the whole TableProvider, and the former may require an expensive catalog operation to construct and the latter may not.

This allows avoiding SELECT * FROM information_schema.tables having to make 1 of those potentially expensive operations per table.

[alamb]

This feature makes sense to me -- thank you @epgif

I wonder if there is some way we can write a test for it (mostly to prevent it from being accidentally broken/changed in the future)

Maybe an example or something 🤔

[epgif]

@alamb

I wonder if there is some way we can write a test for it (mostly to prevent it from being accidentally broken/changed in the future)

I looked around for some tests implementing this interface that I could extend, but didn't find any. I meant to ask earlier but forgot: is there anything like that? It would be a big help if I had some previous tests to take inspiration from.

Failing that, I'll take a stab at it anyway :)

Thanks!

[comphead]

Suggested change

	async fn table_type(&self, name: &str) -> Result<Option<TableType>, DataFusionError> {
	async fn table_type(&self, name: &str) -> DataFusionResult<Option<TableType>> {

[epgif]

I don't see any DataFusionResult in the project, but I do see that E = DataFusionError is the default so I dropped that redundancy. Is that what you meant?

[comphead]

Sorry, messed with another repo. in this file it should be like datafusion_common::Result

in DF we have a type alias in common crate

pub type Result<T, E = DataFusionError> = result::Result<T, E>;

please reuse it

[epgif]

All references to Result in this file are already the one from datafusion_common, including what I've added.

[comphead]

Please change also one in the test

    ) -> Result<Option<Arc<dyn TableProvider>>, DataFusionError> {

[comphead]

is there any way to avoid nested map? perhaps flat_map or and_then ? would me more readable IMHO

[epgif]

Changed to and_then.

[epgif]

Well, clippy suggested what I had to begin with:

warning: using `Result.and_then(|x| Ok(y))`, which is more succinctly expressed as `map(|x| y)`
  --> datafusion/catalog/src/schema.rs:63:9
   |
63 | /         self.table(name)
64 | |             .await
65 | |             .and_then(|o| Ok(o.map(|t| t.table_type())))
   | |________________________________________________________^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#bind_instead_of_map
   = note: `#[warn(clippy::bind_instead_of_map)]` on by default
help: try
   |
63 ~         self.table(name)
64 +             .await.map(|o| o.map(|t| t.table_type()))

So I changed it back.

[epgif]

Please take another look @comphead -- thanks!

[epgif]

@alamb

I wonder if there is some way we can write a test for it (mostly to prevent it from being accidentally broken/changed in the future)

I looked around for some tests implementing this interface that I could extend, but didn't find any.

I tried panic-driven development and identified various unrelated test, none providing a clue.

The trick here is a test needs to check that one function is called rather than another. This seems like a classic low-level unit test.

So, that's what I've done.

It looks noisy if you're not used to this kind of test, but it does work, and does provide the expected failure when the information_schema.rs part is reverted (i.e. wrong function called).

Let me know what you think.

Thanks, @alamb!