Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add short type aliases for schema #30

Open
MaxGraey opened this issue Aug 5, 2022 · 3 comments
Open

Add short type aliases for schema #30

MaxGraey opened this issue Aug 5, 2022 · 3 comments

Comments

@MaxGraey
Copy link
Contributor

MaxGraey commented Aug 5, 2022

I see basic type names inspired by Go type system. But most of modern languages like Rust, zig, AS and WebAssembly WAT btw has a short type notation. So I propose add this aliases:

byte     ->  u8
char     ->  i8
int16    ->  i16
uint16   ->  u16
int32    ->  i32
uint32   ->  u32
int64    ->  i64
uint64   ->  u64
float32  ->  f64
float64  ->  f64

[]char   ->  str
@MaxGraey MaxGraey changed the title Add short type aliases Add short type aliases for schema Aug 5, 2022
@inkeliz
Copy link
Owner

inkeliz commented Aug 7, 2022

I don't think that is a good addition, honestly. That creates more ways of doing the same thing. We could end up with one code like:

struct Foo inline {
A uint32;
B UInt32;
C u32;
D uint32_t;
E uint;
}

I'm supposing that we add support for all types for each language (and common naming). I think there's two alternatives (rather than adding alias):

  1. Work to implement auto-complete/syntax-highlight for most IDEs (JetBrains, VSCode). By consequence, typing i32 will suggest int32, or even replace it.
  2. Work to solve Restrict conflicting names and reserved names #17 first. Then, use some karmem fmt to replace any i32 to int32, and equivalent. The formatter will understand the alias (as you suggest), but not the generator will not understand it. In that case, the original file will be modified, and then any i32, int, Int32, int32_t (...) will be replaced by int32.

The []char was intentional, since you can't do [][]AnyType. If you use str (or string) you may think that []str is possible, which isn't.

@MaxGraey
Copy link
Contributor Author

MaxGraey commented Aug 7, 2022

Thanks for explanation!

you may think that []str is possible, which isn't.

But isn't this pretty meaningful limitation? Why arrays of strings not possible? Or is it temporal limitation?

@inkeliz
Copy link
Owner

inkeliz commented Aug 7, 2022

Currently, you can wrap strings into some inline struct, such as:

struct StringContainer inline {
Data []char;
}

struct Foo table {
Strings []StringContainer;
}

It's possible to do that wrapping "auto-magically", but currently it doesn't.


That limitation is because how that is represented in-memory, which makes possible to use the native arrays from most languages (Go, Zig, C), which also take advantage of some compilers optimizations.

Since inline have constant-size, we can create arrays point to the location and with fixed size. In that case, each StringContainer will always have 16bytes. So, the Strings can pointer to the area which each element will always have 16bytes, so that is possible:

strings := foo.Strings(reader)
for i := 0; i < len(strings); i++ {
    fmt.Print(strings[i].Data(reader))
}

In that case foo.Strings(reader) returns []StringContainerViewer, one "native array". That is not a copy of the array.

However, if foo.Strings(reader) returns []string (native string from the language): that doesn't align-up. Because the string must point to the location in memory (which karmem encoded have another base-pointer), it also needs to have the length in bytes or in another encoder... So, the strings[i].Data(reader) is responsible to create one string (in most cases still near-zero-copy, some languages like AS and C# pays the price of UTF-16 encoding).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants