The following is a lecture by John Newberry. Essentially, it’s a lecture that describes the proofs and mathematics underpinning the calculations for private keys and public keys. I’ve followed along and wrote down everything that has been said here, in order to get a better grasp on elliptic curves, finite fields, etc.
A transaction consists of:
one or more txins, which contain:
a reference to the txout that is being spent
a digital signature proving that the owner of the private key authorized the transaction
one or more txouts, which contain:
the public key of the recipient
Verifying A Transaction
Bitcoin nodes do the following to verify each transaction:
Check that each txin, corresponds with an unspent txout.
Checks that the total amount in the txouts, does not exceed the total amount from the txins.
Check that each txin contains a valid signautre for the public key from the txout referenced.
Digital Signatures are used to transfer ownership of coins.
A digital signature proves that the owner of the coin authorised the transfer:
only someone with the private key can sign the transaction (authentication)
no-one can change the transaction after it has been signed (integrity
What is a digital signature?
Digital signatures make use of asymmetric cryptography.
The user has a public key (which is published and known), and a corresponding private key.
Only someone with this private key can create a valid signature over a message. Anyone with the public key and the message, can confirm that the signature is valid.
Digital Signatures and Bitcoin
Bitcoin uses ECDSA over secp256k1, however a better signature algorithm is the Schnorr signature algorithm, as ECDSA is a bit of a hack to get around the patent reserving the Schorr signature.
ECDSA is an application of the discrete log problem, essentially meaning it is easy to multiply in this system, but much more difficult to divide.
Discrete logs are defined for cyclic groups with a generator G. The problem is defined as:
for a given H in the group, what is the scalar X, such that XG=H
Bitcoin uses the group of points on the elliptic curve secp256k1 defined over a finite field.
Defined as a set of objects along with a binary operator +. The binary operator has the following settings.
Closure If you add two elements together, they are still an element within the group.
Identity Adding 0 to X, results in X
Inverse If you have a number X within a group, you also have the inverse, and when adding these two elements together, it returns 0.
Associativity (a + b) + c = a + (b + c)
Communicativity a + b = b + a
A group is considered cyclic, if there is a generator element:
a = g +g + g … (n times)
This I really didn’t understand, and the more I rewatched this part, the more I got confused. The following Youtube video explained the nature of cyclic groups and helped me understand a lot more about what we’re dealing with here.
Additionally, some of the notation in this video is explained, and it makes sense of the notation in the original lecture.
A field is a commutative group with a secondary binary operatorX.
The second binary operator is also closed, has an identity (1), has inverses (except for 0), is associative and commutative.
The binary operators are also distributive:
a x (b + c) = (a x b) + c
We can add, subtract, multiply, and divide over a field.
The real numbers are thought of as an infinite field.
The rational numbers, with addition and multiplication defined as normal, is an infinite field.
Integers from 0 to (n – 1), with addition and multiplication defined modulo n is a finite field.
Instead of defining secp256k1 over all reals, we define it over a finite field of integers mod p.
Defining a Group Operation for the Elliptic Curve
To add two points:
Take the line meeting the two points
Find where the line intersects the curve again
Reflect through the x axis
To double a point:
Take the tangent at that point
Find where the tangent meets the curve again
Reflect through the X axis
A point’s inverse is the reflection in the X axis.
Adding a point to its inverse, yields our group identity: the point a infinity.
Adding the point at infinity to any point P, yields P.
Generating a Cyclic Group
Take any point G on the curve.
Repeatedly add it to itseflf until you reach G again.
The set of points generated is a cyclic group.
Discrete Log Problem For An Elliptic Curve
The private key, x, is scalar, 256 bit number in the range, [0, .., n-1], where n is the order of the group.
The public key is a point P on the curve where:
P = xG
It is easy to go from x to P.
It is computationally difficult to go from P to x.
This essentially means, it is easy to generate a public key from a private key, but much more difficult to generate a private key from a public key, because division in finite fields is difficult. This is how we know that our public and private keys are secure.
Schnorr Identification Protocol
A prover can prove to a verifier that she knows the private key x corresponding to a public key P without revealing x.
The verified learns nothing about x from the proof (except the fact that the prover knows x).
This is called a proof in zero knowledge.
A zero-knowledge proof requires three properties:
Completeness – the proof convinces the verifier
Zero-knowledgeness – the proof doesn’t leak information
Soundness – a proof can only be produced by a prover who knows the private key
Schnorr Identification Protocol Steps
Commitment – The prover picks a nonce scalar kand commits to it by sending K = kG to the verifier.
Challenge – The verifier sends a challenge scalar e
Response – The prover sends the response scalar: s = k + ex
The verified is convinced that the prover knows x if the identity holds:
sG = kG + exG
= K + eP
The transcript of the 3 step protocol is: (K, e, s)
If you can produce a proof, for any challenge e, then you must know x.
If verifier has:
s1 = k + e1x
s2 = k + e2x
They can now calculate:
x = s1 - s2 / e1 - e2
Non-Interactive Schorr Identification Protocol
The verifier’s only role was to provide a “random” challenge.
If we can replace the verifier with a random oracle that simply provides a random number after the commitment step, then we don’t need a verifier.
We treat a hash function as a random oracle.
“After” is key here. It means that the prover cannot know the output to a hash function before evaluating it.
This is called a Fiat-Shamir transform.
This has 3 steps:
The prover picks a nonce scalar k
The prover calculates e=H(kG)
The prover computes the scalar s = k + ex
The proof is (s,e)
Anyone can verify by checking:
sG = kG + exG
Signature of A Message
Since H is a random oracle and returns different values for different inputs, the prover can add extra inputs to H.
The result is a signature of knowledge over a mesage.
The prover can set:
e = H(m||kg)
The prover calculates s in the normal way:
s = k + ex
The verifier then checks that:
sG = kG + exG
e = H(m||kg)
We use ECDSA with Bitcoin because Schnorr signatures were encumbered by a patent.
Few disadvantages compared to Schnorr:
Signatures are not linear (makes threshold and adaptor signatures much more difficult)
There is no security proof for ECDSA
ECDSA signatures are malleable.
The prover signs a message m as follows:
Set z as the leftmost bits of H(m)
Pick a random nonce scalar k
Set K = kG and r as the x coordinate of K
Set s = k-1(z + rx)
The signature is the pair (r,s)
Verification of ECDSA
The signatures are verfiied as follows:
Set z as the leftmost bits of H()m
Set u = z/s and v = r/s
If the x coordinate of uG + vP is equal to r, then the signature is valid.
What happens to the bitcoin network when the miners all stop in the future? What happens to the bitcoin network when the miners all stop, years in the future after all the bitcoins have been mined? How will the network continue to function? Won’t bitcoins then be useless? What would be the incentive for an individual to continue using computational power to service all the transactions? Isn’t this like a ticking time bomb or is there something I’m not getting?
If I were to rephrase this question, it would be something like this, “What incentives do Bitcoin miners have to keep mining?”
I thought this might be an interesting question to tackle, especially since we just had a halvening today! So let’s dive into it.
Why do Bitcoin Miners mine?
Bitcoin miners earn money, every time they guess a correct answer to a problem.
As more people participate in guessing, the problem gets harder and harder.
If you get a correct answer, you win money. Currently that reward is 6.25 bitcoin. This is why Bitcoin miners mine. To get Bitcoin.
What happens in the future?
As time goes on, the amount of Bitcoin will approach 21 million. When we hit 21 million, there will not be a reward for the miners. Now, if there is no reward for the miners, why would they mine? I don’t work for free. You probably don’t either! The miners probably would feel the same way.
The good news is that Satoshi already thought of this problem. We can pay for miners to accept our transactions, think of it as a tip. So if I send 1 BTC, I’ll add a tip (known as a fee) so it can get accepted by a miner.
A Bitcoin block contains on average around 1600 transactions. With 1600 tips, the miners can replace their previous reward, with tip rewards.
What are the incentives?
To make it very simple and to directly answer our question we asked earlier (remember? We asked, “What incentives do Bitcoin miners have to keep mining?”).
The incentives are simple: Bitcoin miners are motivated by Bitcoin. As the reward rate of Bitcoin goes down, the fees (tips we send on every transaction) go up, replacing the income lost.
Also, as long as Bitcoin maintains and raises in value, it could be considered very valuable to mine. Even if the reward is very little (6.25btc vs the original reward, 50 btc), the 6.25 btc earned now is worth much more than the 50 btc were worth at the time of earning.
Bitcoin miners are incentivized by money. They currently earn money through rewards, and a little bit through fees. In the future they can earn all their money through fees. Also, the price of Bitcoin is very important to the miners, because if the price was $0.00, they couldn’t make very much money from mining the bitcoin.
Today, while I was talking to my barber, I told her about myself, and she used this word, “Ambitious.” to describe me. She only knows me through my presentation, yet she attempts to prescribe a narrative as to who I am.
When I think about people I admire, I think of the elements that I admire; Long-term planning/action, strength and control, insight, genuine relatability, and many more. Most of these, I lack. So it is no wonder, why I admire to these people.
Much like the barber, I think we are all desperate to create someone. Someone who is better than us, who has better abilities than us, who can complete us.
Most attempt to find completion in marriage. Others, search for a father figure or acceptance from a father figure.
Recently I watched “The Two Popes” on Netflix. While it was refreshing to consume media with a spiritual wrapping, the underbelly of the movie seemed to say, “Progressivism is Enlightment”.
This is not an unpopular refrain, in fact, perhaps a bit tired one, to be frank. The idea that “Progressivism” will save us, shows up in social debates about gender, environmental concerns, and political expression.
I think the reason for this, is that conservatism has so deeply failed us.
We tried conservatism, in the form of the Drug War. We were fed that drugs (marijuana) would destroy families and homes. So, those that disobey must be punished. This was the African American communities, the addicts and anyone else who dared to disobey.
The result was nothing. We’ve gained nothing from the Drug War, except the idea that government enforced conservatism is failure.
We also tried conservative flavored war. Following the attacks of 9/11, the government decided that they need to know everything, so that they can protect us. The Patriot Act soon followed, with invasions into the Middle East.
No one knows why we are there. No one knows what we hope to accomplish. No one knows when we will accomplish it. We’ve eroded our own freedoms in the process. But, here we are.
Now, this isn’t a screed against conservatism. This isn’t a sermon praising Progressivism. Progressivism will soon fall, just like its father, Conservatism, as they both rely heavily on the greatest deception of all, Trust In Government.
Our belief in ourselves and the individuals around us, eroded sometime after our belief in God. Now the only thing that has power, seems to be the government, and so in it, we place all power.
Our church-led rituals and personal self-affirmations are replaced by voting and arguments over who the “chosen-one” is to lead (who we only choose through self-identity).
We believe these individuals, once properly ascended on-high, will be endued with power to set everything right, as it should be. We live vicariously through them, knowing that their victory, is our victory; their rule is our rule, only to be betrayed when we realize they are self-actualized beings, fallible in all manners.
The current system loves progressive narratives. We shouldn’t blame it. It used to love conservative narratives, until they failed us.
When progressivism fails, we will not re-examine ourselves. Instead, it will be time to embrace a new story, a new doctrine and a new savior.
Ephesians 4:14) That we henceforth be no more children, tossed to and fro, and carried about with every wind of doctrine, by the sleight of men, and cunning craftiness, whereby they lie in wait to deceive;
So, I will be the first to admit that in this journey of learning about Bitcoin, I have sorta been all over the place. Going from reading books, to reading documentation, to reading articles, to trying to hack something together, to failing, to reading, back to hacking, back to failing, on and on.
This time, I am setting an actual project for myself though. I am going to build a testnet wallet from scratch.
This will be a seriously difficult exercise for me. I am aware. I am giving myself until the end of January to build a BAD bitcoin testnet wallet.
The specifications are as follows:
Generate and store private key
Generate and store addresses
Delete addresses on user request
Delete private keys on user request
Send transactions to other addresses
Initial block download
Restore from Mneonmic
Labels and user management
For me, this is a massive undertaking. All I know right now, is how to generate addresses.
I figure, if I can hack this together in a month + 9 days, I can be considered a bad bitcoin developer. If I can be considered any sort of a Bitcoin developer, that is a WIN in my book.
I will try and report daily progress. I won’t make daily progress, but I definitely will make an effort.
This next week is critical for me since it is Christmas week, and I took the entire week off, so I can spend extra time to read, study, learn and build.
At first, I really didn’t get this at all. I read that it was impossible, but I didn’t understand how.
As I read the mailing list, I got progressively more confused. I really didn’t understand what was being talked about – isStandard, Anyone-can-spend scripts, etc.
But finally, I came across the quote below, and it illuminated it all for me very well.
Since a soft-fork is a restriction of the consensus rules, I think the only way to have an un-soft-forkable cryptocurrency is creating a cryptocurrency where no transaction is valid.
Imagine I build a very minimal cryptocurrency where in the transaction output you only indicate the public key to send your coins to and the amount. One can still soft-fork it by deciding that, from now on, only even amounts are valid or only public keys that are a multiple of 10 are valid.
After I read this I understood the context of all the other elements they were discussing on the mailing list.
Bitcoin is mostly focused on preventing double-spends using proof-of-work.
Proof-of-work is based off of Hashcash.
Hashcash is a poisson process.
A Bitcoin without block reward would end up as a disincentive miners to act honestly.
Confirmations: 51% hashpower isn’t necessary for a short-term attack.
Writing consensus code is interestingly difficult.
The coding to consensus slide was probably the most interesting part of the talk, at least to me.
Great talk – I learned a lot about forking, and the Bad Ideas section was very enlightening.
The thing I really took away from this talk was: Bitcoin is a fragile system, but has a GREAT incentive structure, that takes weaknesses (eg. miners could try and double-spend) into strengths (if they double-spend, it destroys the use case).
Was very interesting to trace Bitcoin back to its original ideas. This paper was not very technical, but was rather a sort of technical recap. Any advanced topics were spelled out quite easily, even something seemingly complicated like Blockchain is described in very simple language.
This is definitely a paper that I will refer back on in the future. There is just so much history, I am sure I missed something.
The most interesting thing I learned was about hashcash. I didn’t realize that the solving of the puzzle itself, was the cash, in this protocol. It was cool seeing how Bitcoin was adopted to include proof-of-work, and the coinage/cash element was a complete separate part of this element.
Also, reading about Blockchains as a separate entity, with the paper evaluating the usage of a Blockchain among a consortium of banks, as the small number of parties would not need Nakamoto consensus.
I’ve read this before, recently actually. It was great to re-read it. You can sense the admiration Ray has for Satoshi. He brings out, in great lengths, the characteristics of Satoshi and how it hasn’t been replicated since.
The un-scammy nature of Satoshi should have set a standard for the entire space, but many people ended up seeing Bitcoin & Blockchain as a get rich quick scheme, and still do.
It’s really exciting to read something through the lens of someone who has been in the Bitcoin space since the beginning. It gave me an interesting perspective on Satoshi, but it seems like Ray is a bit regretful of the whole experience, because of the abuse, but understands that it is a function of human nature.