matrix-rust-sdk/benchmarks
Timo Kösters 2f58cb1620
members: Simplify disambiguation logic when loading member list (#3184)
When all room members are loaded, we do not need an incremental member update. We know that parsing the /members response will only lead to more ambiguous names, not less. And because /members returns the complete list, we can directly use that list as the disambiguation map.

This improves the performance in my emulator from 56s to 9s and on a less performant device from 11mins to 11s (Tested experimentally on Matrix HQ using log statements in element android. If I have time, I will write a proper benchmark tomorrow.

See also https://github.com/matrix-org/matrix-rust-sdk/pull/3184#issuecomment-1986170631 for a more detailed benchmark run.

---

* members: Simplify disambiguation logic

* members: Prevent api misuse for receive_members

* members: Benchmark receive_all_members performance

* sdk: remove unused import

* sdk-base: rename `ApiMisuse` error to `InvalidReceiveMembersParameters`

* benchmarks: extract the member loading benchmark to `room_bench.rs`

* benchmarks: remove wiremock

* sdk-base: fix format

* sdk-base: try fixing tests

* benchmark: Provide some data to the store so the search and disambiguation happen

* benchmark: fix clippy

* benchmark: use a constant for `MEMBERS_IN_ROOM`

* sdk(style): reduce indent in `receive_all_members`

---------

Co-authored-by: Jorge Martín <jorgem@element.io>
Co-authored-by: Benjamin Bouvier <public@benj.me>
2024-03-12 10:15:30 +00:00
..
benches members: Simplify disambiguation logic when loading member list (#3184) 2024-03-12 10:15:30 +00:00
Cargo.toml members: Simplify disambiguation logic when loading member list (#3184) 2024-03-12 10:15:30 +00:00
README.md Move the crypto benchmarks into a separate crate 2022-03-31 14:26:32 +02:00

README.md

Benchmarks for the rust-sdk crypto layer

This directory contains various benchmarks that test critical functionality in the crypto layer in the rust-sdk.

We're using Criterion for the benchmarks, the full documentation for Criterion can be found here.

Running the benchmarks

The benchmark can be simply run by using the bench command of cargo:

$ cargo bench

This will work from the workspace directory of the rust-sdk.

If you want to pass options to the benchmark you'll need to specify the name of the benchmark:

$ cargo bench --bench crypto_bench -- # Your options go here

If you want to run only a specific benchmark, simply pass the name of the benchmark as an argument:

$ cargo bench --bench crypto_bench "Room key sharing/"

After the benchmarks are done, a HTML report can be found in target/criterion/report/index.html.

Using a baseline for the benchmark

The benchmarks will by default compare the results to the previous run of the benchmark. If you are improving the performance of a specific feature and run the benchmark many times, it may be useful to store a baseline to compare against instead.

The --save-baseline switch can be used to create a baseline for the benchmark.

$ cargo bench --bench crypto_bench -- --save-baseline libolm

After you make your changes you can use the baseline to compare the results like so:

$ cargo bench --bench crypto_bench -- --baseline libolm

Generating Flame Graphs for the benchmarks

The benchmarks support profiling and generating Flame Graphs while they run in profiling mode using pprof.

Profiling usually requires root permissions, to avoid the need for root permissions you can adjust the value of perf_event_paranoid, e.g. the most permisive value is -1:

$ echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid

To generate flame graphs feature simply enable the profiling mode using the --profile-time command line flag:

$ cargo bench --bench crypto_bench -- --profile-time=5

After the benchmarks are done, a flame graph for each individual benchmark can be found in target/criterion/<name-of-benchmark>/profile/flamegraph.svg.