User avatar
max
CEO
Posts: 303
Joined: Wed Jan 20, 2016 7:59 pm
Location: Berlin
Contact: Website Twitter

Technical Meeting - October 14th, 2016

Mon Oct 10, 2016 7:16 am

Outlined is a summary of the technical meeting.

This upcoming Friday, October 14th, Lisk will be conducting it’s third community technical meeting. For each meeting, a document is created for gathering questions that may be difficult or lengthy to answer during the meeting. The deadline for collecting questions is Wednesday (October 12th). If you believe your question is not difficult, nor lengthy, you may attend the meeting and ask the team to get a direct answer.

Note: The meetings have changed from the usual Saturday gatherings to Friday. Also, the upcoming technical meeting was postponed one week from its usual day and time. We will continue the community meetings on a two-week basis.

Everyone is encouraged to participate, so if you have any detailed questions for us, please submit them to joel, MrV, MrFrismint, tharude, cc001, GreXX, malreynolds, and/or punkrock within Lisk chat, and they will be added to the document. Alternatively, you can also send the questions through e-mail (joel@lisk.io) During the meeting, the team will address as many questions as possible.

The meeting will take place in our Lisk chat, and will be held on October 14th, 2016 at 6:00 PM (GMT +2).

Technical Meeting Questions

Release-Management/Development-Process
What are the rough next steps/priorities in development for the next 4 weeks? What are the most important things to be addressed/fixed? (cc001)
Oliver: This week we have closed a number of issues related to the mainchain stabilisation milestone. The remaining are two hard issues are concerned with memory table consistency and block propagation efficiency. These I hope to solve within 1–2 weeks and are deemed the most important.

Along with the two issues mentioned above, we have some easy / medium issues still to resolve, but these are less impactful in terms of enabling community forging and applicable to the wider scope of mainchain stabilization.

The next release candidate 0.4.1a will be released to alpha testers over the weekend and includes massive efficiency improvements in the persistence layer of the peers module. We shall need to test these changes, but so far the results are dramatic, and will definitely help weaker performing nodes from forking under heavy traffic conditions.

The issues closed in 0.4.1a are as follows:

Improved peers do db efficiency
https://github.com/LiskHQ/lisk/issues/104 (Hard)

The old peers module had some lingering inefficiencies brought over from the original Crypti code base. Where every peer communication resulted in two database writes, that each used a connection handle from the database connection pool. This was obviously very inefficient and under high traffic conditions would starve the connection pool, lead to high CPU usage, and likely result in a node forking, especially if running a delegate.

The new implementation collects peers changes and sweeps them in batches at a rate of 100 per second, with each batch using only one database query and one handle from the connection pool.

Loading unconfirmed transactions and signatures from “good” peer
https://github.com/LiskHQ/lisk/issues/294 (Medium)

In 0.4.0, every 14 seconds a node will merge its memory pool with the contents from another peer. We have now added some quality control to the peer a node decides to pull this data from.

This should solve the problem recently experienced where a “stuck” nodes with “stale” memory pool were being used to pull from, resulting in a large number already confirmed transactions to be processed every 14 seconds.

Automatically enabling forging after blockchain rebuild
https://github.com/LiskHQ/lisk/issues/141 (Easy)

Forging is now enabled immediately after rebuilding the blockchain (if passphrases are defined within config.json).

Therefore removing the need to reload or restart in order to enable forging on a rebuilt node.

Verify genesis block matches connected database on startup
https://github.com/LiskHQ/lisk/issues/255 (Easy)

The genesis block defined at the file level is now matched against the connected database before further processing. This prevents developers who swap genesis blocks during development won’t harm an existing chain’s integrity by accident.

Added web socket broadcast when there is a fork
https://github.com/LiskHQ/lisk/issues/284 (Easy)

A web socket broadcast is now made when a node enters a fork. Allowing real-time access to the fork data previously only logged or persisted to the database.

Could/Would you change the block time from 10 sec to something >20 sec in the next testnet release as shift did, to improve network stability? (redsn0w)
Joel: Increasing the block time from 10 seconds to 20 seconds is a step backwards. We’re highly against it. The issues concerning block propagation to peers needs to be resolved in order to achieve constant 10s block times. As far as I know, there are two things that blockchain technology needs to improve and these are:
1) block times being too high and
2) transactions per block.

We believe the stability of the network can be achieved with a 10s block time.

Might it be possible to create a validated snapshot from block zero to 1,000,000 and if you make a new snapshot at 1,100,000 you only have to validate the last 100,000 blocks and merge both snapshots into one validated snapshot? This would save much time, if it’s not necessary to validate always from zero anymore. (punkrock)[/b
[b]Isabella
: There is a possibility to implement this functionality in the future if it is proven viable and safe. I have had many discussions about this with Oliver and we both agree that it is possible. It is not a priority to implement currently, since verifying from 0 on an existing database provides us the best data integrity.

All users are welcome to maintain their own snapshots using lisk_snapshot.sh on their own node. With the tool, it is possible to validate a snapshot of the blockchain in uptime using a second instance of lisk. Anyone who wishes to use it should plan accordingly when sizing their node.

Does Lisk use “Practical Byzantine Fault Tolerance”? (wannabe_rotebaron)
Max: I assume you are talking about the consensus algorithm actively developed by the Hyperledger project, right? The day after I met you in Frankfurt I attended a conference with two presentations about PBFT, one by Andreas Fletcher from Deutsche Börse Group and the other one by Thomas Hartmann from IBM Germany. It sounded quite interesting and we will look into it, especially because another cryptocurrency based on the same code base as Lisk is using it. We already contacted them and will see what kind of opportunities arise. Currently, Lisk is not using PBFT. Instead we are using DPoS, originally formulated by BitShares.

Could you give more infos about how the delegates are chosen to create a block, including the important parts in the code? (wannabe_rotebaron)
Oliver: Please take a look at my answer provided in the technical meeting from the 13th August.

How delegates are chosen:

1. Lisk gets a list of all 101 active delegates sorted by vote (code).
2. Lisk determines an order of delegates based on an initial seed (code).
  • The initial seed is a sha256 hash of the current round, derived from block height.
  • Delegates are then indexed according to currentSeed[x] % 101, with x being an individual byte of said sha256 hash (code).
  • For every 4th byte of the current seed, a new hash is created based on the previous one.
3. Lisk determines a slot number according to a given timestamp, and selects the delegates key from the previously generated delegates list
4. Lisk checks if the current slot matches any enabled delegates, if there is an exact match on slot time, the delegate generates and signs a new block

Therefore, although it appears that it is done randomly, the list is generated in a way which can be determined by all nodes in unison, providing their memory tables are in consensus.

Will it be possible in the future to add a short message to a transaction? If yes, when is this planned? (punkrock)
Max: I know this is a very useful feature for exchanges, online shops, and individuals. However, this would bloat up the mainchain quite a bit. We are building Lisk for the future, and in order to keep the mainchain as small as possible we keep the system modular. That means there will very, very likely be a sidechain which allows you to add a short message to a mainchain transaction.

What’s the gGmbH state of play? (LSK)
Joel: The Non-Profit is in progress. During the past couple of weeks, the German government has become quite an obstacle in terms of Lisk (the non-profit) complying with their regulations/requirements. This week we had a call with our lawyers in order to figure out what’s the best and fastest way to move forward. We believe we found a good, alternate solution and will reveal more details as soon as our lawyers give the okay.

Max: Following Joel’s answer we are currently investigating with our lawyers what we can announce in order to keep you updated. Unfortunately, with legal topics it’s always a hassle. With the next general community meeting we will have more topics to talk about.

Where are the talented new devs? It concerns me that finding suitable people is taking so long. Perhaps they are not that interested in Lisk. Is the Lisk brand inspiring, or is it I’ll defined? (LSK)
Joel: We have interviewed a couple of talented developers, but many of these developers have additional offers on more prestigious corporate companies. We are interviewing highly skilled candidates on a weekly basis. Finding a good fit takes some time, as the first few hires are critical to the company’s success.

Max: We are pushing harder on that topic because hiring turns out to be more time consuming and difficult than we imagined. Therefore, we are now conducting 3–4 interviews per week and are ramping up the salaries.

Lisk Clients
So there is the main Lisk client, fix’s lite client, and Ferro’s nano client. Which is preferred to be used by your average user?
Max: Today, this probably would be Lisk Nano. An easy-to-use client which enables you to send around funds. (Which just received an update yesterday!) After the rebranding we will split the front end from the back end, meaning we have a daemon for the core functionalities of Lisk and lite clients for all different user interfaces. This allows us to update these two components individually, and therefore introduce updates faster and more secure.

All in all, long-term it will definitely be the regular Lisk client. More specifically the user interface done for mobile devices. There will be a big jump in usability/accessibility, user experience, and design.

Any comparison done between the lite and nano clients for users to understand quickly the difference?
Max: First of all, the lite client done by Fixcrypt is not official Lisk team software. That means we can’t guarantee its security or contribute to its reliability. Second, we named the nano client “nano” because it has a minimal feature set. Create an account, send and receive money, and being fully responsive. That’s it. The lite client has more features like multiple accounts.

Lastly, is work on the nano client (since fix’s is not official I don’t believe) stalled until the mainnet code is stabilized and optimized, as the github don’t show anything for the last month? If so, what is Ferro working on since his github shows no contributions since Sept. 1? (Long question by MrV)
Max: Lisk Nano was 100% done by Ricardo, here in the chat called rferro, who also did the Lisk paper wallet. Ricardo doesn’t contribute to the core code of Lisk, due to his time constraints. He was in the USA recently to study English, since a few days he is back. Consequently, yesterday he pushed Lisk Nano v0.1.1 which now supports (amongst other things) second passphrases. He is already working on the next version currently.

Security
Have you looked into the code of Asch (or seen the talk in lisk chat)? It seems that they have some nice ideas about consensus and fork prevention. Might be worth a look or a talk with their developer (he’s in Lisk Chat) What do you think about it? (cc001)
Isabella: I have seen the discussion about Asch on Lisk.chat. They offer some wonderful ideas that have merit, such as the propose concept for consensus for block create. There are a few issues though. Their code base is based on a much older version of Lisk, having forked 0.1.4.

If @holdthedoor wants to send us information, we welcome him to do so. Maybe by sharing developments everyone can advance together!

Technical Meeting Transcript

Hey Guys! We’re about to start our Technical Meeting for today. @all*
I would like to say a few things before we begin. First, I hope everyone was excited about the recent initiative of rewarding heavy contributors of the Lisk ecosystem. I must say, these guys really did deserve this, they have done so much for Lisk.

Second, Oliver just pushed quite a few of commits to the mainchain stabilization milestone (https://github.com/LiskHQ/lisk/milestone/2) and now it’s up to 76% with 19 issues closed! Even though there might be some issues added as we find more, it’s sure it great progress that Oliver is making. One step at a time :)

And third, this week we released 0.4.0 to the main network. It’s been great until now, with only a few minor hiccups (including that pesky UI bug with new accounts). There’s also the new version of Lisk Nano which now supports a second passphrase, make sure to check that out.

Thanks for the excellent questions you guys have posted on the document. Please make sure to checkout https://blog.lisk.io/community-technica ... .d5nzxp6c6 as all of the questions have been answered. Hopefully, we’ll have even more great questions during the meeting :)

Alright guys, let’s start this meeting! Please make sure to tag each question to a single member (e.g. @max Is your hair thinning at your age?)

Liskybusiness: Not a technical question, but: When is the complete rebrand scheduled? I wonder if it would help credibility with certain banking or legal relationships. I mean, right now the logo is a massive gemstone and we’re going for non-profit. Not sure if it makes a difference, but just wondering!
Max: Hey @liskybusiness . So we had several meetings with several design agencies this week, and a few more in the past 2 weeks. We are still in the process of finding the best one or the one we like the most. We are getting close. :)

Then for the design itself, it will take about 3–4 months to get a complete rebranding done. Delays not calculated in. After that we would still have to develop the user interfaces, which of course can be done in parallel and sooner.

After going to a lot of events and talking to many investors, regulators, some banks, etc. No one ever moaned about the logo, I think it won’t be a hindrance. Especially, because lately the focus is on the “Lisk” font itself. I will discuss it internally with Oliver and Joel, and see if we can make some changes here sooner.

Forger_of_lsk: Don’t you guys think we have more immediate issues to worry about than re-branding? Don’t you think that can wait, and time spent in other areas?
Max: Not a lot of time is being spent on the rebranding, but you see that it takes a long time to finish. It’s definitely not a priority for us at this moment, however we can spend 2–3 hours a week on these meetings to make progress in that regard. Once the decisions are made the majority of work will be done by the design studio. It’s a parallel effort which allows us to move on faster.

Polycrypto: I’m just sporadically online this morning. Can you give a brief overview of what is the situation in the testnet today?
Oliver: I’m still reviewing what happened but as far as I can tell it was a stress test that lead to a high load being placed on delegates, ending in them forking. The peers efficiency improvements I’ve pushed out today, which I was working on in the past few days, would have largely mitigated this issue.

I’m preparing to release these changes to alpha testers over the weekend.

LSK: What rebranding brief will the design agency receive from you guy…you guys at Lisk?
Max: There’s quite a back and forth process to the whole rebranding design phase. But like @max mentioned, we’re currently shopping and looking at design agencies, we’re not at that stage yet.

Forger_of_lsk: a follow up question on the answer given in the forum with regards to gGmbH. So is gGmbH totally off the table and as I think you were hinting, you and the lawyers are pursuing alternatives?
Max: We are pursuing alternatives. But, we can’t say anything more until our lawyers give their OK.

Doweig: Who are your lawyers currently?
Max: Joel: We can give more details as things progress. Right now we need something more tangible.

Doweig: I’m asking because there’s not so many lawyers who can help with the legal stuff about crypto currencies at the moment. I’m just curious to know who you going through. Are they based in germany?
Doweig: Are you just using regular lawyers? No special crypto knowledge?

Max: Yep, sorry. I just understand what you meant after your edit. We have been in contact with lawyers in America, Germany, and surrounding areas. We are working together with lawyers with a lot of cryptocurrency expertise.

Joel: Ah I see. Yeah, there aren’t many lawyer firms who are up to date with cryptocurrencies, but we’ve been working with some knowledgeable lawyers. We’ve been given a couple of contacts of law firms that have quite a bit of expertise.

wannabe_RoteBaron: I did understand that you @isabella spend a lot of time on writing test for the code that is probably a corner stone for the future development. Do you plan to run by yourself this type of stress test in the furure?
Oliver: I am unable to give any numbers until we test new code, but I think based on changes made we should be able to increase max per block significantly. There was a real bottleneck in the peers stack and we have now removed it I think.

wannabe_RoteBaron: do you need any specific help with the tests for the version that you want to release this weekend?
Oliver: I think we will need help testing sustained maximum throughput of transactions, and vote swapping. These are the most sensitive issues we need to test.

Cc001: the testnet still seems to be very unstable, right now a new split seems to develop. any idea why that happens and how to prevent it?
isabella: the amount of nodes on forks with forging enabled below current height is contributing to the creation of new stub chains

forging should only be enabled once you are fully synced to help prevent this issue

Oliver: I think the new peer selection code introduced in 0.4.0 possibly needs some adjustment / refinement. It is working very well under normal circumstances, but when there is a fork or multiple forks it becomes a problem.

Cc001: maybe we should make new updates mandatory and exclude old nodes from the network to prevent such behavior with buggy old nodes?
Oliver: Yes, we are going to add a minimal version parameter to the Lisk config. This should help eliminate unwanted versions. The problem we faced today was also due to this: https://github.com/LiskHQ/lisk/issues/294. Now fixed.

LSK: Any view on the price slide we are seeing?
Joel: My personal view is: I don’t like looking at the price since I don’t trade LSK, I just buy and buy and keep accumulating.

Ubik: What do you think about SHIFT already being able to stably DPOS forge? From what i understand you don’t want to rush things and you don’t want to give any estimates but can we have nonetheless a shaky “year” estimate?
Oliver: If they are forging stabile it is because of 100s of refactoring and code improvements we’ve made, not Shift. Please remember that.

We are working hard to produce the best possible implementation. Not merely, a mediocre one that gets by.

If you look at peers db efficiency changes we just made, this is just one example of where Shift will surely fall down in terms of stability and scalability. The greatest evidence being what has happened on the testnet today.

Max: Please note that our network is 100x more valuable. We have more responsibility here. Else @oliver said all. I think some of our community members can give a clearer picture how stable they are, because I saw there are some delegates from Lisk also being active at Shift. Can someone shed light on this?

Cc001: is it correct that the peers select the chain based on the number of other nodes at that height?
Oliver: “Good” peers are selected based on a normalized distribution of heights, therefore majority consensus on height wins overtime.

Cc001: also for rebuild I think? I tried to add “good” nodes as seed peers in the config, but it didn’t seem to help.
Oliver: This applies to rebuild / syncing in general. Like I said before, I think there is some adjustment to be made here.

Bitbanksy: Good evening everyone, one quick question: if the German government become a “real obstacle in terms of Lisk (the non-profit) complying with their regulations/requirements”, have you considered the option of setting up a foundation in Switzerland?
Joel: Yes, we considered it. :)

Bitbanksy: Thx @joel can you shed more light on the subject?
Joel: Not really, but a Switzerland foundation seemed easier to make in my opinion. And that’s just from speaking with a few of the ETH guys.

Bitbanksy: No prob @joel but if I may ask again, is the “consideration” your own personal opinion or the team’s one?
Joel: My opinion :).

Bitbanksy: One more question re: testnet… few weeks ago, perhaps a month or so, there was a comment made by one of you guys about the possibility of rewarding/covering expenses of some testnet delegates…. Are they any more update about it?
Joel: No program for running a testnet. It’s a one time testnet bounty.
There’s a bug bounty program in the works which rewards users for finding bugs in the platform.

Bitbanksy: well sorry I used the wrong term… Thx anyway for the clarification.
Joel: No worries, but I had to clarify or else someone would think that we were going to introduce a testnet bounty program for running a node on the testnet.

Bitbanksy: can we have more info on that “one time testnet bounty”? if not when will info be released?
Joel: Not much more info I can give at the moment regarding the bounty, I really haven’t looked at it this week.

Dakk: I have a tech question: currently the main implementation of Lisk is written in nodejs; why did you chose it and why didn’t you chose a more safe language?
Max: I could speak about it all day, but key points are:

- Being the continuation of Crypti, we obviously are using the same code base which is written in Node.js
- Node.js and JavaScript has this huge momentum behind them, there are so many meetups, events, and developers around the world. It’s crazy.
- JavaScript is extremely accessible, it’s running everywhere.
- With JavaScript we can have more rapid development cycles.
- Thanks to TypeScript we can have the same level of type safety as any other strongly typed language. We will be implementing it across the board.


ok that’s was just my curiosity; typescript is a great idea. I would also suggest you to integrate a type-system like flowtype (https://flowtype.org): you can write type annotations directly on comments (so the code remains pure javascript), and the flow typechecker will make the dirty work for you
Max: Cools, thanks. We will take a look and research flowtype. Personally, I never heard about it.
Please take a look at the Community Fund. It's an important project!

Return to “Community Meetings”

Who is online

Users browsing this forum: No registered users and 1 guest