Add support for .NET 7/C# #29

inkeliz · 2022-08-05T01:58:15Z

No description provided.

inkeliz · 2022-08-05T02:02:32Z

Still missing:

Support for Reset()
More tests and fuzzing
Compile to WASM/WASI
Export functions to WASM (C#-only way to declare wasm imports and exports SteveSandersonMS/dotnet-wasi-sdk#30)

inkeliz · 2022-08-05T08:12:37Z

There's also two issues:

C# can't have fields with the same name of the Class, such as:

public struct MyName {
    public uint MyName = 0;
}

That forces the generator to include _ in the name, transforming the field to _MyName. That can be undesirable in must cases. I think the best approach is fixing #17, then emitting warnings when such collisions happens and maybe just append _ to conflicted names.

C# can't hold more than 2GB on array:

The List<T> can only use int as key/index, and that is a limiting factor. Also, far I understand, any managed object can't use more than 2GB. However, unmanaged doesn't have such limits.

Because of that I'm thinking to move all types of array to Karmem.Slice, since it's just one pointer to the data. The 2GB restrictions doesn't apply to

inkeliz · 2022-08-07T05:01:03Z

Currently, the test is faling due to out-of-memory. That issue is already know: SteveSandersonMS/dotnet-wasi-sdk#11.

However, that is might a hint of poor performance and the lack stack-reuse. The code should re-use the space and should prevent additional allocation. It can be either an lack of C# optimization or bug in the current Karmem code. More experimentation soon.

dotnet/Karmem.cs

inkeliz · 2022-08-07T10:02:36Z

benchmark/dotnet/Program.c

+__attribute__((export_name("InputMemoryPointer")))
+uint32_t InputMemoryPointer() {
+    MonoObject* err;
+    MonoObject* res = mono_wasm_invoke_method (_ID_InputMemoryPointer, _CLASS_Benchmark, NULL, &err);


How release that pointer? 🧐

inkeliz · 2022-08-07T17:58:40Z

Looking into Span internals, seems that is optimized by JIT, which might explains why karmem.Slice is ~5ms slower. I'm not C# expert, but sounds like there's some internal code optimized and I'm not sure if there's any equivalent of go:linkname from Go on C#, allowing to call internal functions directly.

Currently, benchmark for Vec3 is ~75ms (and ~70ms using Span), on Ryzen 3900X (M1 tests soon).

That still significantly slower than Go, Zig, C, AS and even Swift (with #11 optimization). I'm not sure what I can do to optimize it, and sounds like issues on C# runtime directly.

inkeliz · 2022-08-08T08:05:14Z

Ok. Compiling without -c Release reveals the issue, and it's releated to the lack of GC (or lack of GrowCalls):

[wasm_trace_logger] * Assertion at /home/runner/work/dotnet-wasi-sdk/dotnet-wasi-sdk/modules/runtime/src/mono/mono/metadata/sgen-stw.c:77, condition `info->client_info.stack_start >= info->client_info.info.stack_start_limit && info->client_info.stack_start < info->client_info.info.stack_end' not met

    main_wasi_fuzz_test.go:126: wasm error: unreachable
        wasm stack trace:
                ._Exit(i32)
                .exit(i32)
                .wasm_trace_logger(i32,i32,i32,i32,i32)
                .eglib_log_adapter(i32,i32,i32,i32)
                .monoeg_g_logstr(i32,i32,i32)
                .monoeg_g_logv_nofree(i32,i32,i32,i32) i32
                .monoeg_assertion_message(i32,i32)
                .mono_assertion_message(i32,i32,i32)
                .update_current_thread_stack(i32)
                .sgen_client_stop_world(i32,i32)
                .sgen_stop_world(i32,i32)
                .sgen_perform_collection_inner(i32,i32,i32,i32,i32)
                .sgen_perform_collection(i32,i32,i32,i32,i32)
                .sgen_ensure_free_space(i32,i32)
                .sgen_los_alloc_large_inner(i32,i32) i32
                .sgen_alloc_obj_nolock(i32,i32) i32
                .mono_gc_alloc_vector(i32,i32,i32) i32
                .mono_array_new_specific_internal(i32,i32,i32,i32) i32
                .mono_array_new_specific_checked(i32,i32,i32) i32
                .interp_exec_method(i32,i32,i32)
                .interp_runtime_invoke(i32,i32,i32,i32,i32) i32
                .mono_jit_runtime_invoke(i32,i32,i32,i32,i32) i32
                .do_runtime_invoke(i32,i32,i32,i32,i32) i32
                .mono_runtime_try_invoke(i32,i32,i32,i32,i32) i32
                .mono_runtime_invoke(i32,i32,i32,i32) i32
                .mono_wasm_invoke_method_ref(i32,i32,i32,i32,i32)
                .mono_wasm_invoke_method(i32,i32,i32,i32) i32
                .KBenchmarkDecodeObjectAPI(i32)

inkeliz · 2022-08-10T08:13:21Z

The tests are passing on TCP/Native, the Fuzzing is falling on WASM. However, the Flatbuffers is also failling on WASM, while working on TCP/Native.

I'll assume that it's an issue of DotNet-WASI-SDK (SteveSandersonMS/dotnet-wasi-sdk#11), however it might be one issue from Karmem itself.

This patch adds support for .NET 7.

That change makes possible to use uint/ulong as index-key. However, since C# doesn't have `usize`, we need to use `ulong`.

Before that patch, Karmem.Slice doesn't support other value rather than Karmem.IViewer.

Before that patch, the value was retrieved by copy. That prevents modifications of non-primitive values doesn't reflects to the original List<T>.

That implements the Reset function, which is present for all generated code.

Before that patch, new allocations are not cleared

That changes makes possible to compile to WASM with the standard benchmark exported functions.

Now it's possible to compile and test/bench with `make dotnet`.

… instead of opaque pointers Previously, the Viewer struct is one opaque pointer to the data. Now, that was changed to "hold" the content and be able to use unsafe to cast it directly, instead of having one temporary variable. The performance improves from ~104ms to ~89ms. The struct uses `Unsafe.AsPointer(this)` to get their own pointer.

Previously, we remain check for implementation of IViewer. However, since last patch it's no longer necessary and we can remove the interface. That also improves the performance from ~89ms to ~75ms.

That reverts the #37a7496 commit, however Viewers is also supported.

Now, the generated code uses Span.

Previously, the size of the string was wrongly calculated and defined.

That change makes clear that Viewers are not suppose to be modified.

Some languages are failing the fuzzing for reasons outside our control.

That makes possible to test without need WebAssembly

To compile for WASM you need to provide `-p:IS_WASM=true` building tags

Previously, the Flatbuffers and Karmem are included, even when not used.

inkeliz commented Aug 7, 2022

View reviewed changes

dotnet/Karmem.cs Outdated Show resolved Hide resolved

inkeliz commented Aug 7, 2022

View reviewed changes

inkeliz force-pushed the lang-dotnet branch from 18add05 to 0a8cc69 Compare August 7, 2022 10:30

inkeliz force-pushed the lang-dotnet branch 5 times, most recently from 29c5431 to 35e31d3 Compare August 9, 2022 19:04

inkeliz force-pushed the lang-dotnet branch from ed8b712 to 8b6726f Compare August 10, 2022 08:21

inkeliz added 15 commits August 15, 2022 23:11

add: dotnet package

44ab28d

This patch adds support for .NET 7.

kmgen: [dotnet] add intial code generation support

62e8537

benchmark: [dotnet] add initial benchmark code

dc9e161

kmgen: remove Span<T> in favor of Karmen.Slice<T>

065f5fc

That change makes possible to use uint/ulong as index-key. However, since C# doesn't have `usize`, we need to use `ulong`.

dotnet: add support for primitive values on Slice

0d96cff

Before that patch, Karmem.Slice doesn't support other value rather than Karmem.IViewer.

kmgen: get List<T> values by reference

ea503b6

Before that patch, the value was retrieved by copy. That prevents modifications of non-primitive values doesn't reflects to the original List<T>.

dotnet: use long instead of uint

0fb4213

kmgen,benchmark: [dotnet] implement Reset method

cf2dd0d

That implements the Reset function, which is present for all generated code.

dotnet: fix memset

b521816

Before that patch, new allocations are not cleared

benchmark: [dotnet] add exported functions with Mono trampoline

baa953b

That changes makes possible to compile to WASM with the standard benchmark exported functions.

benchmark: [dotnet] add wasm compilation and test

03722ba

Now it's possible to compile and test/bench with `make dotnet`.

dotnet: remove deadcode and rename private fields

3a68084

dotnet,kmgen: [dotnet] improve performance by removing IViewer interface

177ab4d

Previously, we remain check for implementation of IViewer. However, since last patch it's no longer necessary and we can remove the interface. That also improves the performance from ~89ms to ~75ms.

kmgen,benchmark: [dotnet] remove Karmem.Slice in favor of Span<T>

d98e801

That reverts the #37a7496 commit, however Viewers is also supported.

inkeliz added 22 commits August 15, 2022 23:23

dotnet: remove Karmem.Slice struct

1709116

Now, the generated code uses Span.

benchmark: [dotnet] change ulong to int

4324155

benchmark: [dotnet] add dotnet.package directive

cde0da7

dotnet,kmgen: [dotnet] fix writer

fa63bca

dotnet,kmgen: [dotnet] fix string writer and encoder

256ba8c

Previously, the size of the string was wrongly calculated and defined.

kmgen: [dotnet] simplify reset

b460a70

kmgen: [dotnet] replace Span<T> to ReadOnlySpan<T>

0388375

That change makes clear that Viewers are not suppose to be modified.

benchmark: add new nofuzz build-tag to ignore fuzzing

bc5ec40

Some languages are failing the fuzzing for reasons outside our control.

benchmark: [c,dotnet] update generated code and makefile

ed6256e

ci: add dotnet compilation and tests

1da5ff5

benchmark: add new nobench build-tag to ignore benchmark

84acc68

ci: [dotnet] run tests without benchmark

594edf2

kmgen,benchmark: [dotnet] attempt to reduce allocations

941b370

benchmark: add test interface via TCP

55ca0a5

That makes possible to test without need WebAssembly

benchmark: [dotnet] add support for TCP

71b705b

To compile for WASM you need to provide `-p:IS_WASM=true` building tags

benchmark,ci:[dotnet] force wasm build

fa46cce

ci,benchmark: [dotnet] add Flatbuffers

fd4ecda

benchmark: [dotnet] use conditional build tags for includes

537b08b

Previously, the Flatbuffers and Karmem are included, even when not used.

benchmark: code cleanup

f79088e

benchmark: [dotnet] fix macos compilation

4622bc3

readme: add C#/.NET information

36f5f7b

benchmark: cleanup Makefile

5d6e97d

inkeliz force-pushed the lang-dotnet branch from f00cbaf to 5d6e97d Compare August 15, 2022 22:30

inkeliz merged commit 3b20cee into master Aug 15, 2022

inkeliz mentioned this pull request Aug 23, 2022

Support for .NET 7 #14

Closed

inkeliz deleted the lang-dotnet branch August 29, 2022 00:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for .NET 7/C# #29

Add support for .NET 7/C# #29

inkeliz commented Aug 5, 2022

inkeliz commented Aug 5, 2022 •

edited

Loading

inkeliz commented Aug 5, 2022 •

edited

Loading

inkeliz commented Aug 7, 2022

inkeliz Aug 7, 2022

inkeliz commented Aug 7, 2022 •

edited

Loading

inkeliz commented Aug 8, 2022

inkeliz commented Aug 10, 2022

Add support for .NET 7/C# #29

Add support for .NET 7/C# #29

Conversation

inkeliz commented Aug 5, 2022

inkeliz commented Aug 5, 2022 • edited Loading

inkeliz commented Aug 5, 2022 • edited Loading

inkeliz commented Aug 7, 2022

inkeliz Aug 7, 2022

Choose a reason for hiding this comment

inkeliz commented Aug 7, 2022 • edited Loading

inkeliz commented Aug 8, 2022

inkeliz commented Aug 10, 2022

inkeliz commented Aug 5, 2022 •

edited

Loading

inkeliz commented Aug 5, 2022 •

edited

Loading

inkeliz commented Aug 7, 2022 •

edited

Loading