How to derive hashrate from hash difficulty

Hey everyone! I've been testing out the hashrate for some miners in a bitcoin app, and ran into a bit of a snag: The hashrate calculated based on the difficulty of the shares submitted is way off compared to the actual hashrate of the miners, like a few times different. This happens with both ASICs and CPUMiner. Here's how we’re figuring out the hashrate (just some pseudocode to keep it simple): ``` diff = max_target / hash total_hashes = sum(difficulty * pow(2, 32)) hash_rate = total_hashes / sec ``` So basically, we’re calculating how many hashes are needed to hit that difficulty (`difficulty * pow(2, 32)`), then we take all the hashes found over the last 10 minutes, add them up, and divide by 600 seconds. Seems straightforward, right? But in reality, CPUMiner is finding shares with difficulty levels all over the place, like from 0.01 to 10.0 in those 10 minutes. Because of that, our formula gives us a hashrate that swings wildly, often way off from what it actually is. Has anyone dealt with something like this? Any ideas on how to fix it or where to look next? Thanks a lot for any help!

Isn't that normal? That's why the network hashrate is an estimate, and the difficulty is only adjusted every 2016 blocks. Let me start by saying I'm not a Bitcoin miner, but can't you increase the time period over which you calculate the hashrate?

We are developing a mining pool, and it would be beneficial for us to know the hashrate statistics of each worker over a 10-minute period to distribute rewards fairly. Essentially, we need the hashrate displayed by the miner's device to match the hashrate we calculate based on the difficulty of the shares (hashes) it submits.

Can't you accomplish that by making the shares small enough, so the miner has multiple shares per reward period? That way you should get a nice average instead of peaks.

A real-world example: during local testing with CPUMiner, 50% of the shares come in with a difficulty between 0.05 and 0.15 at a rate of 10 shares per minute. However, episodically, CPUMiner calculates shares with a difficulty of 10.0 and even 40.0 - and this completely skews the hashrate calculation over 10-minute intervals. I've tried trimming extreme values in the calculations, but I still can't achieve acceptable accuracy between the hashrate reported by CPUMiner in its logs and the hashrate we calculate. I'm currently experimenting with different normalization schemes, but haven't found a suitable solution yet. So I thought - maybe someone has a ready-made solution.

Shouldn't each share have the same difficulty?

From my understanding of mining as it is a completely random process what determines the hashrate of a worker is the frequency with which the worker can deliver valid shares that are above a certain lower difficulty threshold. The pool tells the worker (gives work items) to report back only if it finds a share above difficulty x. This threshold x has to be chosen based on the worker's capabilites and is subject to be adjusted on-the-fly. The worker shouldn't have to report back too often, with too high frequency (work items have too low difficulty for worker's hashpower) and also not with too low frequency as the pool then might not know if the worker is actually doing some work. I think it's a misconception to calculate the hashrate based on the specific difficulty value of a submitted share. You should observe the frequency with which a certain difficulty is exceeded, regardless of the specific value that exceeded the lower limit. If I'm not wrong (see https://en.bitcoin.it/wiki/Difficulty) to statistically exceed difficulty D in timeframe 600s you have to execute a hashrate of at least D * 232 / 600. To sample with higher frequency you can adjust accordingly. The mining pool code of ckPool is open-source. Last time I looked at the code, I don't remember it very well documented. You may have a look at the code, if you haven't done so already. Repository link is https://bitbucket.org/ckolivas/ckpool

We are working with an engine based on https://github.com/stratum-mining/stratum - you are probably right, it's worth basing calculations on the target rather than the average hash_diff.

That is not how mining pools normally work. Normally, the mining pool creates it's own target value that is (for example) 232 * T, where T is the current target. A miner submits all hashes less than the pool target. These are the "shares". The pool may use the rate of submitted shares to estimate a miner's hash rate, but the hash rate value is only informational. The number of shares is what is used to determine the payout according to the pool's distribution system. I suggest taking a look at the Mining forum.