Skip to content

Generate fields in REPL for data schemas #867

Open
@Jolanrensen

Description

@Jolanrensen

#866 reverted the PR Generate fields in REPL for data schemas since it caused a bug in schema generation in notebooks. However, this feature is still needed for the IntelliJ Plugin of DataFrame to be able to extract the schema from DataFrame types.

In short, enabling generating fields = true makes it visible that the data schema generation makes a mistake with inheritance.

Originally

@DataSchema
interface _DataFrameType1 {}

val ColumnsContainer<_DataFrameType1>.columnA: DataColumn<DataFrame<_DataFrameType10>> 
  get() = this["columnA"] as DataColumn<DataFrame<_DataFrameType10>>

@DataSchema
interface _DataFrameType2 : _DataFrameType1 {}

val ColumnsContainer<_DataFrameType2>.columnA: DataColumn<DataFrame<_DataFrameType20>>
  get() = this["columnA"] as DataColumn<DataFrame<_DataFrameType20>>

is not an issue as long as _DataFrameType10 and _DataFrameType20 have the same properties.

However, when the properties are generated

@DataSchema
interface _DataFrameType1 {
  val columnA: List<_DataFrameType10>
}

@DataSchema
interface _DataFrameType2 : _DataFrameType1 {
  override val columnA: List<_DataFrameType20> // wrong type
}

becomes a problem. This only works if _DataFrameType20 is a subtype of _DataFrameType10, which is not the case in the current generation implementation.

Reproducible test by @koperagen:
image
image

@koperagen:

i found the culprit: @DataSchema(isOpen = false)
not sure about the fix yet. but it goes like this: technically _DataFrameType is a supertype of _DataFrameType1 according to structure, so nothing is wrong here. But then code generator doesn't make Repos1 : Repos because isOpen = false

Maybe making isOpen = true solves the issue, but this needs to be properly tested for 0.15. Alternatively, fields = false could remain as is for notebooks, however, this is not desired as the IntelliJ plugin might have some use in notebooks in the future.

Metadata

Metadata

Labels

bugSomething isn't workingresearchThis requires a deeper dive to gather a better understanding

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions