why does cgminer update "nonce2" and "ntime" every 5 seconds?

Hey everyone, I’ve got some questions about how mining works with cgminer. I’ve done a bit of reading on the topic. Here’s a snippet showing cgminer mining on the mainnet (connected to bitcoind @localhost) using a GekkoScience Compac F USB miner at an average hash rate of 387 GH/s. I’ve trimmed some of the long numbers for easier reading. The nonce range is from 0 to 2^32-1 (0-4294967295), which we can call 4.3 billion. If I’m right, this USB miner, with a hash rate of 387 GH/s, should be able to go through the entire nonce range in about 1/90th of a second. So, that means it could theoretically cycle through the whole nonce range 90 times a second. Am I on the right track? If I get it, the process works like this after the block header is created: - The miner tests the whole nonce range starting from 0 and going up to 4294967295. - If it doesn’t find a hash below the target, then nonce2 (or extranonce) can be adjusted. This is just some random data that slightly tweaks the coinbase transaction, changing the Merkle root. At this point, there are millions more chances to find a valid hash. If that still doesn’t work... - you can also change the ntime. This is the time in seconds. The ntime goes up by one each second. You can adjust ntime locally, but there are limits; you can’t just move it around freely because the chain won’t accept that. I’ve heard somewhere that a range of plus or minus 10 minutes is allowed. Is that true? I couldn’t find that info quickly.

You can adjust the ntime binary data in submit_noffset_nonce or resubmit to be able to pull the stratum split every 5 seconds to 2 minutes. You can see here https://bitcointalk.org/index.php?topic=28402.0, for more clarity.

Hi Snowy Pear, thank you for creating an extra new account to reply to me. I looked into submit_noffset_nonce, but I just couldn't manage to change ntime variation to a different value every 5 seconds or turn it off briefly for testing. I even tried test turning off all of the following functions by commenting those functions: modify_ntime, roll_work, roll_work_ntime and everywhere else where noffset occurs (including the references to it, of course). After a "make" and restarting the program in debug mode, the ntime is still incremented every 5 seconds. I don't understand your quoting. I hope you don't confuse my question with the 5 sec which is the default setting for page rendering? Because cgminer automatically refreshes the status page every 5 seconds. But that's not my point at all. I related my question exclusively to the counting up ntime. Where does it get this info, if I have disabled everything in cgminer.c regarding noffset? And why 5 seconds, where does it get this value = 5sec ?

NTime is just a unix timestamp. You can take the 63667f00 or 63667f05 ntimes, convert from hex to decimal and put it into https://www.unixtimestamp.com/ to check what dates those refer to. I suppose cgminer creates a new block template with updated timestamp only once every 5 seconds to reduce overhead; especially since the timestamp in Bitcoin's block header doesn't have to be very accurate, anyway.

I know that it represents the timestamp in UNIX time format. But I would like to understand where in the program code the 5 seconds are pulled from. I can't find the corresponding place in cgminer.c EDIT: Meanwhile while exploring the debug log, I have recognized that not only ntime is being flipped each 5 seconds but also the incrementing nonce2 is being reset after the 5 secs. Where does the 5 seconds come from ? any help appreciated. EDIT2: SOLVED Found it! It comes from block in line 9364

Yup, it makes sense that the nonce is reset once you have a new block template with updated timestamp, right? I'm happy you found the code segment, though; but it leaves me curious what you need it for. Are you trying to mess with the 'reset timeout'? If so, why?

Hi n0nce, it is just for education purposes, trying to understand and fiddling in the code.

In cgminer,  any new block or any update of cuurent block will create a new "ntime" when block information is received.  About 60 second, block update information is coming from pool. see parse_notify() function. In cgminer, nonce2 is used to create more local block header data for more hashing and is used for generating new merkleroot in local header data. it changes very quick. see gen_stratum_work or gen_solo_work funct. It is used in submit work to pool and pool uses it to re-generate block header for verifying if block header is resolved.

Hi jack1cryptotalk007, yes, correct. I am working on gen_solo_work because I'm not using a pool, I'm mining on my own bitcoind node. I'm aware of the reason nonce and nonce2 are used. It is very well explained here and here. Sure? I am talking about the nonce2 (extranonce in coinbase) and not the original nonce. If nonce2 would be incremented, where would be the upper limit of the range ? When I decrement nonce2 in cgminer it starts at 18446744073709551615 so that's why I think it is the upper limit of nonce2. Note: Not to confuse with Extranonce1 vs. Extranonce2 Nope. That is what cgminer seems to do by default. After the 5sec (default) new ntime is generated, nonces are resetted to zero and the process is started over.

Oops, my bad; I'm blaming my tired eyes. No no, I asked whether you changed the code such that it doesn't reset the nonces to zero after 5s. Because somehow in this quote, your timestamp was not updated in 5 whole minutes, right. So you did fiddle with the code. Now, what about my suggestion that the variable may be wrapping around? That's something to consider when working with large numbers / too large numbers for the variable types in use.

I am trying to understand and see live what exactly happens in the mining process. Yes, I did "freeze" ntime for that educational purpose in the example above. I did not freeze nonce2 but I increased the timer from default to 5sec to testwise 15sec, just to see if my changes took effect. So far so good, however I am still far away from my goal. What I like to see is: - all the nonces that cgminer sends to the hardware for work, I like to see how this process is threaded and queued by cgminer - the exact shares that are generated and sent to the hardware - the exact nonce, nonce2 and ntime value that were used for each particular hash step and the result for it Generally spoken, I'm looking to more infos that I miss with the --debug option of cgminer. Currently I'm stuck even trying to display simply the nonce values and the particular hash result of one single step. I am aware that this could output very high amount of lines and generate a log with huge size, but I'm interested in one complete cycle through this: - ntime should be freezed at all, irrelevant for me, same for merkle root - cycling through the whole nonce range and when nonce reached the upper limit the nonce should freeze its value or set to zero and freeze its value (no more incrementing) - then it should cycle through the whole nonce2 range (nonce is fixed at 0) - when nonce2 is exhausted cgminer should exit I wanna see ALL the hash results in a convenient output view (e.g table), so I can scroll through the log file and see each single step including the particular values for nonce and nonce2 and what hashed they generated. note: timestamps are just random examples picked by me, I did not exaclty calculate how long it takes how ?

Add debug prints! Something like this: Then pipe stdout to a file. If that's not good enough, you will need to open a file and write to it instead, but that will be slightly more complicated.

I did so But as I said, I am even struggling in getting the nonce value displayed. I have changed the status bar to see which nonce currently was processed but I get "0" displayed all the time for the nonce. I also tried %u which will give similar error. I am not a developer, can only read some code. I don't think that the format %"PRIu32" or %u is the cause for the issue getting nonce=0 displayed. But I rather suspect that the reason is that in this code section the nonce was not defined at all or not intended to be used. I think that the nonce far before has been sent to the mining hardware as (scheduled/queed) work. Therefore the display of 0 in the output, kinda ?

To be honest, I haven't looked that deep into cgminer yet; if nonce is always 0, I suppose it's possible that it only rotates through nonce2 instead - not sure. Maybe we can summon @kano; he knows for sure.

After I read your posts, there are missing points. 1. The nonce2 is only used to create next new local block header data. When a new block header come in, nonce2 is initialzed to 0 in cgminer, e.g. parse_notify() or gbt_decoode(). The nonce2 is 64 bits integer (very large integer).  The nonce2 only affects merkleroot value in local block header data. Huge numbers of local block header data are created and are given to ASIC chips for chips to find out some nonces which produce a hash value with last 8 digits 0.   hash value: xxxx....x????00000000. 2. cgminer does not cycle nonce from 0 to 4.29G.  It just sends a lot of block header data to ASIC chips. ASIC chips or hardware do cycle nonces by digital chips and frequency pulse to check and find qualified nonces. ASIC hardware do filter out qualified nonces. So, ASIC hash header data super fast. 3. Qualified or found nonces, then, are read by cgminer from ASIC chips. Then, cgminer verify those nonces with the pool's difficulty. Then, good nonces are sent to the pool for further verification by pool with network's difficulty.

Very well explained, thank you. Let's assume nonce2 would be also an unsigned 32 bit integer and the ASIC miner very powerful with a superior hash rate. What would happen when nonce2 range is exhausted before a new block header arrived ? Does that mean that cgminer has no control at all about the current nonce which should be tested? As cgminer sends a block header to the ASIC miner, the ASIC hardware would need to know how such a block header is constructed and to derive the part where the nonce is located, wouldn't it? Otherwise the ASIC miner would only see a very long string (blockheader) without knowing which bits to change for trying for a valid hashes. So cgminer only sends a block header to the ASIC hardware and the ASIC miner itself cycles through the 32bit nonce range? Does the ASIC miner use a random cycle or incrementing the values?

You don't really want cgminer, which runs on a relatively slow CPU to have to increment the nonce and communicate it to ASIC. That's happening much too fast. Chips like the Bitmain BM1387 or the BM1397 (like in your Compac F) have a set of address pins. If I understand this right, the address pins are used when you have several chips in a chain they can each be set to work on a different nonce range, so as not to overlap. cgminer doesn't exactly send a block header to the mining chip. It's like a partially computed hash of the block called a midstate. Kano has a description of this here; https://kano.is/BMMMP63278.pdf. All of the bytes are sent in fixed positions to the mining chip, so it knows what goes where. Things start to get pretty crazy (to me) when cgminer sends multiple midstates to the ASIC, which is called ASICboost.

Thanks for this great explanation and linking to the document. Does this explain why nonce2 traversal is so slow in comparison to nonce, because nonce2 cycling is kept in CPU ? Would it be possible to send nonce2 to the ASIC chip just like as with nonce so the traversal would take place on the much faster ASIC chip? Just as an example: - set nonce to a fixed value e.g. 123 - generate midstate work for nonce2 (instead of nonce) - let the ASIC chip traverse through 2^64-1 cycles for nonce2 and have the hash reported back to cgminer How can that be done ?

Alas there seems to be a lot of misunderstanding reading the code that says exactly what it does. While the code shows it pretty straight forwardly, if you read it, what mining 101 on an asic does is: Receive work from the pool every 30 seconds (that's up to the pool, but 30 seconds is optimal) During that 30 seconds, that single pool work is used over and over again to generate work for the asics. How that 'over and over' is done is by modifying 'nonce2' to generate a unique merkleroot to generating a unique block header. Then hashing that block header doing the first half of the double sha256 then passing that to the asics with a job id. 'nonce' has nothing to do with this part, cgminer has nothing to do with the nonce ranges. A CPU core can hash about 30 million times a second, so generating 10's or 100's or even thousands of work items isn't that hard to do quickly. Each work it generates, will give the asics up to 4billion hashes to do, so as long as the asics aren't 4billion times faster than the time it takes cgminer to produce a single work item, all is good. N.B. a single work item is not one hash, read the code if you want to work out what it does. Using the work sent to them, the asics run the nonce value some % of the way from 0 to 4billion doing the 2nd half of the double sha256 for each nonce value, and if they find a hash with a difficulty of 1 or more, they return the job id and the nonce value that produced the hash. cgminer then tests this returned data with the work job it should match, and sends it to the pool if the difficulty meets the pool's requirements. It does not matter what the asic nonce range is. The only point of interest is for the asics to test a high % of the range to minimise work I/O to the asic. (hashing a nonce range in CPU would be beyond ridiculously slow, millions of times slower than a current asic, so no one would be silly enough to do that ...) That's the basic 101 version of it. Multi midstates, asicboost, and fpga work generation (like in an avalon) requires different effort, to effectively get similar results.

if cgminer has absolutely nothing to do with the nonce  then who does? does the hardware miner have a routine to run through the nonce range - is it triggered by cgminer in some way? weird