Searching for reliable fee estimation APIs

I've connected to several APIs for retrieving transaction and address history, but it's kinda pointless without knowing what fee rate to use. I just need the average fee, not the highs or lows, and I want multiple APIs for backup. I get that fees can vary a lot between different services, so if that happens, I’ll just take the median of all the fee rates I get.

The list of fee estimators that I can find, probably they could help as I know you might have come across some of them before. 1. https://mempool.space/docs/api/rest 2. https://bitcoinjobs.com/fee-calculator 3. https://whatthefee.io/ 4. https://jochen-hoenicke.de/queue/#BTC%20(default%20mempool),24h,weight 5. https://bitcoinfees.net/

Check these ones out too https://bitinfocharts.com/https://www.coingecko.com/api/https://bitcoiner.live/https://dashboard.bitcoinops.org/d/TDhboADmk/bitcoin-optech-dashboards?orgId=1

Why do you need many APIs? Is this mempool.space fee estimation does not provide you with enough information about the fee? Mempool.space is the only source that we always use when checking recommended fees other sources would not give you a good fee estimation compared to mempool.space It works just fine for me I use this https://mempool.space/docs/api/rest#get-recommended-fees Just adding this if you still looking for more fee recommendation API here https://api.blockchain.info/mempool/fees

The larger the datas collected and used,  the closer it is to accuracy. Mempool is the most used, no question about that But sometimes the data it provides might be slightly off from your node's. This might be as a result of configuration Can't really tell.  To add to the list https://jochen-hoenicke.de/queue/#0,24h .

My guess is redundancy. If one api fails OP has others to fall back on and still run the average with decent accuracy or without throwing an error over a dumb cors violation or something unexpected.

I recall you run Bitcoin Core. Have you used estimatesmartfee with various parameter (e.g. estimate_mode) to get some data? As long as you don't use poor data source. I recall in past there was website which always extremely high fee which considered as overpay.

This is old but I think it is worth mentioning Electrum the protocol. Electrum nodes are a better source regarding reliability to get the fee rate in a more decentralized way too. The only problem is that when you are using it without the Electrum software, you have to get familiar with the protocol and write some code yourself. It doesn't look that complicated though, specially since there is a documentation: Just the fee rate https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html#blockchain-relayfee The histogram of mempool: https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html#mempool-get-fee-histogram

Including the Electrum package as a dependency in any serious project will create all sorts of dependency hell, which has the potential to cause a supply chain attack, so it looks like reverse-engineering the protocol is the only option to use that. It doesn't help that the Electrum docs, at least on ReadTheDocs, are hopelessly outdated. I'm pretty sure some network request is being transmitted with each method call, so I could just look at the code for that.

There is some discussion about fee estimation in Bitcoin Core: https://delvingbitcoin.org/t/mempool-based-fee-estimation-on-bitcoin-core/703/1 Some people wrote some code for testing new estimations, so maybe you can use that. Because even if you have some SPV node, then it can still download for example N last blocks, or get some traffic into local mempool, and do some estimations in a similar way.

I have always wondered how Bitcoin Core and other wallets/platforms calculate the average transaction fee from the last N (5?) blocks as well as the economical fee, high priority fee, etc. Do they just look at the getblocktemplate and take the lowest fee paid by a transaction? Or maybe the one in the middle? The raw mempool does contain information on both the fees and sizes of each transaction. (No, I haven't looked into CBlockPolicyEstimator yet...)

By organizing transactions in their mempool into different fee buckets and see which transactions or buckets are being mined and use that with a time decay. This allows Core to estimate a confidence rate of fees for recommendation. This gist explains better than I can: https://gist.github.com/morcos/d3637f015bc4e607e1fd10d8351e9f41. The PR mentioned in that was merged in 0.15.0. IIRC the rationale of not using mempool as an estimate was to provide a more conservative estimate. I think if you want to, it can be improved but with some tradeoff so I'm not sure if its worth it: https://github.com/bitcoin/bitcoin/issues/27995. Anyhow, there is a trend with the fees, or at least a pattern in the frequency that people are sending transactions. It'll definitely be a fun ML project to take on.

Is it really necessary? Mempool.space is just the best fee estimador.  You will add data for redundancy,  however this will have an opposite effect, as you are adding worse data. There are bad estimators out there, and most of them tend to overestimate feerate.

I guess mean/median averaging is a bad idea but you can't say that backup providers are not necessary in case mempool.space has a temporary outage, don't you think?

Unquestionable - I highly advocate for some backup providers for those rare instances that the main one is down. I seem to recall that you also contributed to the discussion (and creation) of FeeBuddy[1] by bitmover and I thought that it would also be nice to have a secondary backup in case mempool.space goes down for a while. Per the code here[2] it seems that there isn't a backup solution placed, but I could be wrong. This website[3]also mentions a couple of free and public API's for the same purpose (some of them were already posted). Perhaps choosing the ones that get most closest to the values that mempools.space provides? [1]https://bitcointalk.org/index.php?action=profile;u=3601165 [2]https://github.com/bitmover-studio/bitcoindata/blob/main/api/bitcoinfees.php [3]https://b10c.me/blog/003-a-list-of-public-bitcoin-feerate-estimation-apis/

I agree that it is useful for backup purposes... Sadly Feebuddy is still down due to my aggressive ddos protection setup. I wasn't able to find a reasonable solution to make it work yet..

This may be a very rudimentary idea, but how about hosting it in a separate service from bitcoindata.science? Since it is anchored in your main website, I reckon the need to have a robust DDoS to guarantee the most uptime. By isolating just that particular "feature", would it make sense to lower the DDoS on that specific domain? Do you think that it makes sense separating these two "services"?

Technically, this is the perfect solution.  I would move all the api to that other machine ,something like api.bitcoindata.science Thee problem would be to pay for that I guess. Maybe @examplens can share his thoughts.  As I use shared server in which he is the admin

It would be ideal if you had at least two or more mirror servers, but it is quite expensive for an unprofitable hobby project. However, even that does not necessarily guarantee trouble-free operation if a targeted DDoS appears on your service. I was thinking about a similar solution earlier, but due to the lack of knowledge in the implementation of API things, and the time to devote to learning, it was put on hold. The idea was to add some kind of historical fee chart to the preev.net domain, where you could call a fee info for a specific timestamp or block, through the API. (Tracking the price of Bitcoin as well). You did it well with bitcoindata.science, so it demotivated me a bit considering that there is a solution. What I would like differently is for it to be textual and not to generate an image. (I have a reason for that) As a potential solution, maybe you could include one of the file hosting services, some of the free packages might even be enough. A bot that will collect certain information and record it on such a server, and api.bitcoindata.science delivers it further. I'm not sure how feasible this is, but I've seen similar solutions in other categories. Keep in mind that in this case, api.bitcoindata.science is only a subdomain, and for example, the DDoS that happened to you was directed only at your domain (not server or hosting IP), considering that the other domains with which you share the server were not affected by the attack. Right now I can't give more than this server, maybe in the near future yes, but for now, I don't want to promise something that I can't guarantee to deliver.

This is easily doable. You can use free API like this one from mempool.space that will give you all the data you need for the fee chart: or this other one: Price chart is also easy to get for free, there are many options around. I even made one myself, if you want I can share with you. price since 2010 , 1 data per day.