Requests are undeniably unmissable for anyone willing to start in the Python ecosystem. Yet, the community is broadly unaware of how sad the situation has become. The Incident Due to dramatic circumstances, it was partially locked and put in "maintenance mode" only, meaning it would no longer evolve (features speaking). Some of you may have heard that more than six years ago, Requests was working toward a major version, namely the V3. There was even r with a substantial amount, around $30k. a completed fundraise At the time, there was a plan to bring Async and HTTP/2 support (most notably). You'll guess that none of the exposed promises ever landed. Impacts This incident led to several impacts. Maintainers felt too much pressure to make steps toward a major; some of them said: . "We just can't deliver the promises, and it's going hard to explain why" All the back pressure landed at urllib3. Abruptly started to refuse any changes, saying: "Requests is feature complete, and it's popularity makes it nearly impossible to fix bugs that may be considered feature to this point." Newer projects started ditching HTTP client Requests by learning to use a new one. Some large-scale projects bound to Requests did not migrate because of the potentially high cost of doing it. Let's be clear, this article is not about saying "Requests maintainers are bad" but rather saying we don't agree, and everyone is entitled to their choices. For many years now, Requests have been frozen. Being left in a vegetative state and not evolving, from using more advanced features. this blocked millions of developers The Journey Into Waking Up Initially, like many other reasonable people, I said to myself "Let's wait, Requests will eventually unblock the situation!" Yet, the years are still passing, and nothing. Let's realize something, Yet, Requests is unable to do so. Internet Explorer 11 just celebrated its tenth anniversary last year, and you’ll never guess that it has support for HTTP/2 integrated! I said to myself: "Why are we waiting for this indefinitely? They don't owe us anything! There's a reasonable chance we are waiting... trapped in a circular reference… Let's act." Then, I tried to get a firm grip on the urllib3 base code, contributing and until I was ready to kick things up with that would have put urllib3 far ahead. Without any breaking changes. I was delusional. This was a bit of a shock, but six months passed between my initial kick-off and , and here's why in a nutshell: this there a proof of concept my formal give-up Every bit of idea brought to the table was dismissed, and never seriously analyzed. This felt as good as taking the coldest wind on your face while exiting your warm home. It's the hard truth of the OSS; it's not a democracy, and it's not ruled by pure logic, but by humans. Not far later on, urllib3 called out to help with HTTP/2 support (only) and claimed around $40k to be able to deliver. They probably meant well, but[...] a fundraiser This was the last straw for me, knowing the past mistakes, and actually the best incentive to break the cycle. Everything points to another year before having the slightest sign of a into our production pipes. They clearly lack the time to move forward (or incentives? or both?). No worries, as developers, we should share this burden for the greater good. PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n From my perspective (my opinion only), the OSS ecosystem should not fundraise money for a decade-old protocol, and most of the time should deliver prior to asking. Entities raising money with no solid proof of concept; that's rare! This felt like a bit of a hostage situation, but it could only be me seeing it this way. The Solution So many years and various expertise have been brought to Requests and urllib3… Do we throw all of it into the garbage disposal? Engineers should be reasonable scientists and not revisionists. We don't have to reinvent the wheel all over again; HTTP client Requests are established and really pleasant in their usage. We believe that Requests has the most inclusive and developer-friendly interfaces. So, how to propose an attractive continuity for Requests? How hard could this be to (i) bring a true upgrade experience and (ii) keep the compatibility as high as possible? Sometimes, the OSS ecosystem acts like it is trying to land a rover on the Moon, let's be more realistic and bring our vision down to firm ground. There's a chance offered to us to rehabilitate Requests (and urllib3) and give a fresh breath of air to us, the developers. Without having to learn something as basic as handling an HTTP client. So, the work started, without looking back. Just around my intent to give up on urllib3 evolution, we proposed a fork of both urllib3 and Requests, namely and . urllib3-future Niquests They are (truly) drop-in replacements, at the time of writing, those forks are six months old! And the whole work around this is roughly a year old. We listened to what the community needed by... triaging all the issues on major repositories (Requests, urllib3, and dependants of them) so that the upgrade would be truly useful. Are you seeing it coming? Great! against ! Let's look at the feature table comparison requests, httpx, and aiohttp Feature niquests requests httpx aiohttp HTTP/1.1 ✅ ✅ ✅ ✅ HTTP/2 ✅ ❌ ✅ ❌ HTTP/3 over QUIC ✅ ❌ ❌ ❌ Synchronous ✅ ✅ ✅ ❌ Asynchronous ✅ ❌ ✅ ✅ Thread Safe ✅ ✅ ❌ N/A Task Safe ✅ N/A ✅ ✅ OS Trust Store ✅ ❌ ❌ ❌ Multiplexing ✅ ❌ Limited ❌ DNSSEC ✅ ❌ ❌ ❌ Customizable DNS Resolution ✅ ❌ ❌ ✅ DNS over HTTPS ✅ ❌ ❌ ❌ DNS over QUIC ✅ ❌ ❌ ❌ DNS over TLS ✅ ❌ ❌ ❌ Multiple DNS Resolver ✅ ❌ ❌ ❌ Network Fine Tuning & Inspect ✅ ❌ Limited Limited Certificate Revocation Protection ✅ ❌ ❌ ❌ Session Persistence ✅ ✅ ✅ ✅ In-memory Certificate CA & mTLS ✅ ❌ Limited Limited SOCKS 4/5 Proxies ✅ ✅ ✅ ❌ HTTP/HTTPS Proxies ✅ ✅ ✅ ✅ TLS-in-TLS Support ✅ ✅ ✅ ✅ Direct HTTP/3 Negotiation ✅ N/A N/A N/A Happy Eyeballs ✅ ❌ ❌ ✅ Package / SLSA Signed ✅ ❌ ❌ ✅ And you would think... performance..? are bad? against ! Look at the performance comparison them Fetch a thousand requests using 10 tasks or threads, each with a hundred requests using a single pool of 10 connection. Scenario: High-Level APIs Client Average Delay to Complete Notes requests 987 ms ThreadPoolExecutor. HTTP/1.1 httpx 735 ms Asyncio. HTTP/2 niquests 470 ms Asyncio. HTTP/2 Simplified APIs Client Average Delay to Complete Notes requests core 643 ms ThreadPoolExecutor. HTTP/1.1 httpx core 550 ms Asyncio. HTTP/2 aiohttp 220 ms Asyncio. HTTP/1.1 niquests core 210 ms Asyncio. HTTP/2 Did you give up on HTTP/2 due to performance concerns? Think again! open up a wide range of possibilities! Multiplexing and response laziness And everything you are witnessing is in fact "Requests"-extended, and you have nothing to learn again to get started. We kept 98% of the test cases from both Requests and urllib3, then extended them to cover newer features. We are looking forward to seeing you at the Niquests repository! Let's say it's a promotional offer, get everything and more for $̶4̶0̶k̶  $0! #1 You depend on Requests? Discover instead. Niquests Drop-in replacement, simple patch needed, tiny effort required. #2 You depend on urllib3? Discover instead. urllib3.future Drop-in replacement, no patch required, atom-like sized effort required. By visiting the GitHub repository and documentation, you will be able to learn more about this, and hopefully make a decisive switch!

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

The Promise That Was Made Six Years Ago for Requests 3 Is Revived

Too Long; Didn't Read

Ahmed TAHRI

STORY’S CREDIBILITY

Opinion piece / Thought Leadership

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

The Promise That Was Made Six Years Ago for Requests 3 Is Revived

Too Long; Didn't Read

Ahmed TAHRI

STORY’S CREDIBILITY

Opinion piece / Thought Leadership

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES