Hacker Newsnew | past | comments | ask | show | jobs | submit | kyotov's commentslogin


Awesome improvements, congrats!


We have released a v1.1 with improved performance!

In summary, we see:

- 10%-100%+ performance improvement with v1.1 on 2 GiB data and 16 threads;

- A significantly improved single thread performance, now on par with Zsync for 2GiB data.

Checkout the whole write-up at https://kyall.notion.site/KySync-v1-1-dd9931f330f241469d3e60...


Instead of an explanation, we decided to address some of the reasons :)

https://kyall.notion.site/KySync-v1-1-dd9931f330f241469d3e60...

Enjoy!


I should have made my intentions clearer from the start :)

KySync started as a pet project to brush up on my C++ skills which I had not used for 20 years and was going to need for a job that I started last week! I had spent my fair share of time optimizing single threaded performance back in grad school, and was looking for different experiences.

That said, and as we mentioned in the write up, we had some improvements cooking that we did not put in v1.0. I spent some time tonight to merge them in and run some performance experiments.

Single threaded performance is now much improved and practically on par with Zsync. I released it as v1.1.

https://kyall.notion.site/KySync-v1-1-dd9931f330f241469d3e60...

We will of course keep looking for more performance, but if you have any suggestions for further improvements, please share -- or join us!

Best, Kamen

P.S. Both of these are contributed by Chaim Mintz, whom I worked with in a previous life.



Indeed... but thanks for the feedback.

Since the content of the page is pretty stable now, I can probably host it in more normal way than Notion.


I was also somewhat frustrated by notion's UI choices and wonder if as a quick hack you could get it into markdown and shove it in a gist (trying to optimise for "annoy me less with as little effort as possible for you" here ;)


Yup! Will be looking at that one of the following nights.

It is so convenient to edit in Notion, but I share the frustration about paging and navigation for reading!


Thanks for the pointer -- I will look into comparing against bita!


If there are specific scenarios that need to be optimized for the use case, please get in touch!


It's not really apples-to-apples as rsync does work on both the client and the server.

Zsync has a comparison with rsync in isolated cases where it makes sense and it is very close in performance!

http://zsync.moria.org.uk/paper/ch02s08.html


We should make a clarification... The intent of KySync (as well as Zsync) is to use across systems, not on a single system. KySync supports HTTP (like Zsync) as well as HTTPS (which Zsync does not).

The primary reason to do the performance comparison on a single system, is so that the results are easy to replicate with as little setup as possible. Because we do this for both KySync and Zsync it is apples to apples.

HTTP bandwidth and storage cost money, and this is self funded project, so I can't afford to put up test files of data publicly visible to the world.

One thing we can look into is leverage AWS/S3 to upload some data and use it for a performance experiment, but that will need some logistics for the developer to set their AWS account properly. Will look into it.

Of course, the more similar the files are, the closer the remote results will be to this first set we published.


So, are you claiming a 3x-10x performance improvement...

* Within the same system?

* Over close-by systems on a LAN segment?

* To typical far machines over the Internet?


I can see your point. We really need to run some experiments in S3 to satisfy this point rigorously.

Intuitively, the more similar the files are, the more 3x-10x+ will be representative in the real world. As the files become more and more different, one is of course at the mercy of the bandwidth between the computers. If you need to transfer 1 GiB differences over 1 MiB/s connection, it will take ~1000 seconds -- no magical way around it.

The comparison so far is practically on the computational cost of the sync, which can be significant as differences pile up.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: