Iteration Starts Again After Expection Python

The library presented in this article is condign obsolete and un-maintained. I recommend you to read this post well-nigh tenacity instead.

I don't oftentimes write about tools I utilize when for my daily software development tasks. I recently realized that I really should start to share more frequently my workflows and weapons of choice.

One matter that I have a hard time indelible while doing Python lawmaking reviews, is people writing utility code that is not straight tied to the core of their business. This looks to me as wasted time maintaining code that should be reused from elsewhere.

And then today I'd similar to get-go with retrying, a Python package that you can employ to… retry annihilation.

Information technology'due south OK to neglect

Often in computing, yous have to deal with external resources. That means accessing resources you don't control. Resource that can fail, go flapping, unreachable or unavailable.

Most applications don't deal with that at all, and explode in flight, leaving a skeptical user in front of the computer. A lot of software engineers refuse to deal with failure, and don't bother handling this kind of scenario in their code.

In the best case, applications ordinarily handle simply the case where the external reached arrangement is out of order. They log something, and inform the user that it should try once more later.

In this cloud computing area, we tend to blueprint software components with service-oriented architecture in listen. That ways having a lot of dissimilar services talking to each others over the network. And nosotros all know that networks tend to neglect, and distributed systems too. Writing software with declining being role of normal operation is a terrific idea.

Retrying

In order to help applications with the handling of these potential failures, you need a plan. Leaving to the user the burden to "try again afterwards" is rarely a practiced choice. Therefore, almost of the fourth dimension you want your application to retry.

Retrying an activity is a full strategy on its own, with a lot of options. You tin retry just on sure status, and with the number of tries based on time (e.one thousand. every 2nd), based on a number of tentative (eastward.grand. retry 3 times and abort), based on the problem encountered, or even on all of those.

For all of that, I utilize the retrying library that you lot tin retrieve hands on PyPI.

retrying provides a decorator called retry that you can use on height of any function or method in Python to make it retry in case of failure. By default, retry calls your part endlessly until it returns rather than raising an error.

              import random from retrying import retry  @retry def pick_one():     if random.randint(0, x) != 1:         enhance Exception("1 was not picked")                          

This will execute the function pick_one until 1 is returned past random.randint.

retry accepts a few arguments, such as the minimum and maximum delays to use, which as well can exist randomized. Randomizing delay is a good strategy to avert detectable pattern or congestion. Merely more over, information technology supports exponential delay, which can be used to implement exponential backoff, a good solution for retrying tasks while really fugitive congestion. It'due south peculiarly handy for background tasks.

              @retry(wait_exponential_multiplier=thousand, wait_exponential_max=10000) def wait_exponential_1000():     print "Wait 2^x * k milliseconds between each retry, up to ten seconds, so x seconds subsequently"     raise Exception("Retry!")                          

You can mix that with a maximum filibuster, which can requite you a good strategy to retry for a while, so fail anyhow:

              # End retrying after 30 seconds anyhow >>> @retry(wait_exponential_multiplier=1000, wait_exponential_max=10000, stop_max_delay=30000) ... def wait_exponential_1000(): ...     impress "Look 2^10 * 1000 milliseconds betwixt each retry, up to 10 seconds, so 10 seconds afterwards" ...     heighten Exception("Retry!") ... >>> wait_exponential_1000() Wait 2^x * thou milliseconds between each retry, upwardly to 10 seconds, then 10 seconds afterwards Wait 2^x * 1000 milliseconds between each retry, up to x seconds, then 10 seconds afterwards Await 2^x * 1000 milliseconds between each retry, upwards to 10 seconds, then 10 seconds afterward Await 2^ten * 1000 milliseconds betwixt each retry, up to 10 seconds, then x seconds later Wait 2^x * 1000 milliseconds between each retry, upwards to 10 seconds, so x seconds later Look two^ten * 1000 milliseconds between each retry, up to ten seconds, then 10 seconds later on Traceback (almost recent call last):   File "<stdin>", line 1, in <module>   File "/usr/local/lib/python2.seven/site-packages/retrying.py", line 49, in wrapped_f     return Retrying(*dargs, **dkw).call(f, *args, **kw)   File "/usr/local/lib/python2.7/site-packages/retrying.py", line 212, in call     raise attempt.get()   File "/usr/local/lib/python2.7/site-packages/retrying.py", line 247, in become     six.reraise(cocky.value[0], self.value[1], self.value[ii])   File "/usr/local/lib/python2.seven/site-packages/retrying.py", line 200, in phone call     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)   File "<stdin>", line 4, in wait_exponential_1000   Exception: Retry!                          

A pattern I use very oftentimes, is the ability to retry simply based on some exception type. You lot tin can specify a function to filter out exception you want to ignore or the ane you desire to use to retry.

              def retry_on_ioerror(exc):     return isinstance(exc, IOError)  @retry(retry_on_exception=retry_on_ioerror) def read_file():     with open("myfile", "r") as f:         return f.read()                          

retry will call the function passed every bit retry_on_exception with the exception raised every bit offset argument. It's up to the function to and so render a boolean indicating if a retry should be performed or not. In the case to a higher place, this will only retry to read the file if an IOError occurs; if any other exception type is raised, no retry will exist performed.

The same pattern tin can exist implemented using the keyword statement retry_on_result, where y'all tin can provide a function that analyses the event and retry based on it.

              def retry_if_file_empty(result):     return len(result) <= 0  @retry(retry_on_result=retry_if_file_empty) def read_file():     with open("myfile", "r") as f:         return f.read()                          

This example will read the file until it stops being empty. If the file does not be, an IOError is raised, and the default behavior which triggers retry on all exceptions kicks-in – the retry is therefore performed.

That's information technology! retry is really a good and small library that yous should leverage rather than implementing your own half-baked solution!

lunahaventruckew.blogspot.com

Source: https://julien.danjou.info/python-retrying/

Share this post