Change history¶
4.0.1¶
- release-date
2019-07-17 3:42 P.M PST
- release-by
Ask Solem (@ask)
Utils:
stampede
objects can now be read by Sphinx andinspect.signature()
.CI: Adds CPython 3.7.3 to build matrix, and set as default for lint stages
4.0.0¶
- release-date
2019-05-07 2:21 P.M PST
- release-by
Ask Solem (@ask)
100% Test Coverage
Fixed several edge case bugs that were never reported.
3.2.2¶
- release-date
2019-04-07 6:18 P.M PST
- release-by
Ask Solem (@ask)
AsyncGenerator
takes two arguments.Mistakenly had it take three arguments, like
typing.Generator
.S
3.2.1¶
- release-date
2019-04-07 4:07 P.M PST
- release-by
Ask Solem (@ask)
Adds
mode.utils.typing.AsyncGenerator
to importtyping.AsyncGenerator
missing from Python 3.6.0.
3.2.0¶
- release-date
2019-04-06 11:00 P.M PST
- release-by
Ask Solem (@ask)
Adds
Service.itertimer
: used to perform periodic tasks, but with automatic drift adjustment.Adds
mode.utils.mocks.ContextMock()
To mock a regular context manager.
3.1.3¶
- release-date
2019-04-04 08:41 P.M PST
- release-by
Ask Solem (@ask)
mode.utils.worker.exiting
now takes option to print exceptions.Threads: Method queue “starting…” logs now logged with debug severity.
Worker: execute_from_commandline no longer swallow errors if loop closed.
Adds
mode.locals.LocalStack
.
3.1.2¶
- release-date
2019-04-04 08:37 P.M PST
- release-by
Ask Solem (@ask)
Revoked release: Version without changelog entry was uploaded to PyPI. Please upgrade to 3.1.3.
3.1.1¶
- release-date
2019-03-27 10:02 A.M PST
- release-by
Ask Solem (@ask)
Service: property
should_stop
is now true if service crashed.Timers: Avoid drift + introduce a tiny bit of drift to timers.
Thanks to Bob Haddleton (@bobh66) for discovering this issue.
3.1.0¶
- release-date
2019-03-21 03:26 P.M PST
- release-by
Ask Solem (@ask)
Adds
nullcontext
andasyncnullcontext
.Backported from Python 3.7 you can import these from
mode.utils.contexts
.Mode project changes:
Added bandit to CI lint build.
Added pydocstyle to CI lint build.
3.0.13¶
- release-date
2019-03-20 04:58 P.M PST
- release-by
Ask Solem (@ask)
Adds
CompositeLogger.warning
alias towarn
.flake8-logging-format has a rule that says you are only allowed to use
.warning
, so going with that.
3.0.12¶
- release-date
2019-03-20 03:23 P.M PST
- release-by
Ask Solem (@ask)
Adds
all_tasks()
as a backward compatibleasyncio.all_tasks()
.Signal: Fixes
.connect()
decorator to work with parens and withoutSignal decorator now works with parens:
@signal.connect() def my_handler(sender, **kwargs): ...
and without parens:
@signal.connect def my_handler(sender, **kwargs): ...
Signal: Do not use weakref by default.
Using weakref by default meant it was too easy to connect a signal handler to only have it disappear because there were no more references to the object.
3.0.11¶
- release-date
2019-03-19 08:50 A.M PST
- release-by
Ask Solem (@ask)
Adds ThrowableQueue._throw() for non-async version of .throw().
3.0.10¶
- release-date
2019-03-14 03:55 P.M PST
- release-by
Ask Solem (@ask)
Worker: was giving successful exit code when an exception is raised.
The
.execute_from_commandline
method now always exits (its return type istyping.NoReturn
).Adds
NoReturn
tomode.utils.compat
.Import
typing.NoReturn
from here to support Python versions before 3.6.3.
3.0.9¶
- release-date
2019-03-08 01:20 P.M PDT
- release-by
Ask Solem (@ask)
Threads: Add multiple workers for thread method queue.
Default number of workers is now 2, to allow for two recursive calls.
Signal: Use Signal.label instead of
.indent
to be more consistent.Signal: Properly set
.name
when signal is member of class.Adds ability to log the FULL traceback of
asyncio.Task
.Service: Stop faster if stopped immediately after start
Service: Correctly track dependencies for services added using
Service.on_init_dependencies
(Issue #40).Contributed by Nimi Wariboko Jr (@nemosupremo).
3.0.8¶
- release-date
2019-01-25 03:54 P.M PDT
- release-by
Ask Solem (@ask)
Fixes
DeprecationWarning
importing fromcollections
.stampede: Fixed edge case where stampede wrapped function called multiple times.
Calling the same stampede wrapped function multiple times within the same event loop iteration would previously call the function multiple times.
For example using
asyncio.gather()
:from mode.utils.futures import stampede count = 0 @stampede async def update_count(): global count count += 1 async def main(): await asyncio.gather( update_count(), update_count(), update_count(), update_count(), ) assert count == 1
Previously this would call the function four times, but with the fix it’s only called once and provides the expected result.
- Mocks: Adds
mask_module()
and
- Mocks: Adds
CI: Added Windows build.
CI: Enabled random order for tests.
3.0.7¶
- release-date
2019-01-18 01:12 P.M PDT
- release-by
Ask Solem (@ask)
ServiceThread
.stop()
would wait for thread shutdown even if thread was never started.CI: Adds CPython 3.7.2 and 3.6.8 to build matrix
3.0.6¶
- release-date
2019-01-07 12:10 P.M PDT
- release-by
Ask Solem (@ask)
Adds
%(extra)s
as log format option.To add additional context to your logging statements use for example:
logger.error('Foo', extra={'data': {'database': 'db1'}})
3.0.5¶
- release-date
2018-12-19 04:40 P.M PDT
- release-by
Ask Solem (@ask)
Fixes compatibility with colorlog 4.0.x.
Contributed by Ryan Whitten (@rwhitten577).
3.0.4¶
- release-date
2018-12-07 04:40 P.M PDT
- release-by
Ask Solem (@ask)
Now depends on mypy_extensions.
3.0.3¶
- release-date
2018-12-07 3:22 P.M PDT
- release-by
Ask Solem (@ask)
Threads: Fixed delay in shutdown if
on_thread_stop
callback raises exception.Service: Stopping of children no longer propagates exceptions, to ensure other services are still stopped.
Worker: Fixed race condition if worker stopped before being fully started.
This would lead the worker to shutdown early before fully stopping all dependent services.
Tests: Adds
AsyncMagicMock
3.0.2¶
- release-date
2018-12-07 1:14 P.M PDT
- release-by
Ask Solem (@ask)
Worker: Fixes crash on Windows where signal handlers cannot be registered.
Utils: Adds
shortname()
to get non-qualified object path.Utils: Adds
canonshortname()
to get non-qualified object path that attempts to resolve the real name of__main__
.
3.0.1¶
- release-date
2018-12-06 10:20 A.M PDT
- release-by
Ask Solem (@ask)
Worker: Added new callback
on_worker_shutdown
.Worker: Do not stop twice, instead wait for original stop to complete.
Signals would start multiple stopping coroutines, leading to the worker shutting down too fast.
Threads: All
ServiceThread
services needs a keepalive coroutine to be scheduled.Supervisor: Fixed issue with
CrashingSupervisor
where service would not crash.
3.0.0¶
- release-date
2018-11-30 4:48 P.M PDT
- release-by
Ask Solem (@ask)
ServiceThread
no longer usesrun_in_executor
.Since services are long running, it is not a good idea for them to block pool worker threads. Instead we run one thread for every ServiceThread.
Adds
QueuedServiceThread
This subclass of
ServiceThread
enables the use of a queue to send work to the service thread.This is useful for services that wrap blocking network clients for example.
If you have a blocking Redis client you could run it in a separate thread like this:
class Redis(QueuedServiceThread): _client: StrictRedis = None async def on_start(self) -> None: self._client = StrictRedis() async def get(self, key): return await self.call_thread(self._client.get, key) async def set(self, key, value): await self.call_thread(self._client.set, key, value)
The actual redis client will be running in a separate thread (with a separate event loop). The
get
andset
methods will delegate to the thread, and return only when the thread is finished handling them and is ready with a result:async def use_redis(): # We use async-with-statement here, but # can also do `await redis.start()` then `await redis.stop()` async with Redis() as redis: await redis.set(key='foo', value='bar') assert await redis.get(key='foo') == 'bar'
Collections:
FastUserSet
andManagedUserSet
now implements allset
operations.Collections are now generic types.
You can now subclass collections with typing information:
class X(FastUserDict[str, int]): ...
class X(ManagedUserDict[str, int]): ...
class X(FastUserSet[str]): ...
class X(ManagedUserSet[str]): ...
maybe_async()
utility now also works with@asyncio.coroutine
decorated coroutines.Worker: SIGUSR1 cry handler: Fixed crash when coroutine does not have
__name__
attribute.
2.0.4¶
- release-date
2018-11-19 1:07 P.M PST
- release-by
Ask Solem (@ask)
FlowControlQueue.clear
now cancels all waiting forQueue.put
.
2.0.3¶
- release-date
2018-11-05 5:20 P.M PDT
- release-by
Ask Solem (@ask)
Adds Service.wait_first(*coros)
Wait for the first coroutine to return, where coroutines can also be
asyncio.Event
.Returns
mode.services.WaitResults
with fields:.done
- List of arguments that are now done..results
- List of return values in order of.done
..stopped
- Set to True if the service was stopped.
2.0.2¶
- release-date
2018-11-03 9:07 A.M PST
- release-by
Ask Solem (@ask)
Now depends on aiocontextvars 0.2
2.0.1¶
- release-date
2018-11-02 7:38 P.M PST
- release-by
Ask Solem (@ask)
Now depends on aiocontextvars 0.2
2.0.0¶
- release-date
2018-11-02 9:12 A.M PST
- release-by
Ask Solem (@ask)
Services now create the event loop on demand.
This means the event loop is no longer created in Service.__init__ so that services can be defined at module scope without initializing the loop.
This makes the
ServiceProxy
pattern redundant for most use cases.Adds
.utils.compat.current_task
as alias forasyncio.current_task
.Adds support for contextvars in Python 3.6 using aiocontextvars.
In mode services you can now use
contextvars
module even on Python 3.6, thanks to the work of @fantix.
1.18.2¶
- release-date
2018-11-30 6:23 P.M PDT
- release-by
Ask Solem (@ask)
Worker: SIGUSR1 cry handler: Fixed crash when coroutine does not have
__name__
attribute.
1.18.1¶
- release-date
2018-10-03 2:49 P.M PDT
- release-by
Ask Solem (@ask)
Service:
Service.from_awaitable(coro)
improvements.The resulting
service.start
will now:Convert awaitable to
asyncio.Task
.Wait for task to complete.
then
service.stop
will:Cancel the task.
This ensures an
asyncio.sleep(10.0)
within can be cancelled. If you need some operation to absolutely finish you must use asyncio.shield.Utils:
cached_property
adds new.is_set(o)
method on descriptorThis can be used to test for the attribute having been cached/used.
If you have a class with a
cached_property
:from mode.utils.objects import cached_property class X: @cached_property def foo(self): return 42 x = X() print(x.foo)
From an instance you can now check if the property was accessed:
if type(x).foo.is_set(x): print(f'Someone accessed x.foo and it was cached as: {x.foo}')
1.18.0¶
- release-date
2018-10-02 3:32 P.M PDT
- release-by
Ask Solem (@ask)
Worker: Fixed error when starting aioconsole on
--debug
The worker would crash with:
TypeError: Use `self.add_context(ctx)` for non-async context
when started with the
--debug
flag.Worker: New
daemon
argument controls shutdown of worker.When the flag is enabled, the default, the worker will not shut down until the worker instance is either explicitly stopped, or it receives a terminating process signal (
SIGINT
/SIGTERM
/etc.)When disabled, the worker for the given service will shut down as soon as
await service.start()
returns.You can think of it as a flag for daemons, but one that doesn’t actually do any of the UNIX daemonization stuff (detaching, etc.). It merely means the worker continues to run in the background until stopped by signal.
Service: Added class method:
Service.from_awaitable
.This can be used to create a service out of any coroutine or
Awaitable
:from mode import Service, Worker async def me(interval=1.0): print('ME STARTING') await asyncio.sleep(interval) print('ME STOPPING') def run_worker(interval=1.0): coro = me(interval=1.0) Worker(Service.from_awaitable(coro)).execute_from_commandline() if __name__ == '__main__': run_worker()
Note
Using a service with
await self.sleep(1.0)
is often not what you want, as stopping the service will have to wait for the sleep to finish.Service.from_awaitable
is as such a last resort for cases where you’re provided a coroutine you cannot implement as a service.Service.sleep()
is useful as it will stop sleeping immediately if the service is stopped:class Me(Service):
- async def on_start(self) -> None:
await self.sleep(1.0)
- Service: New method
_repr_name
can be used to override the service class name used in
repr(service)
.
- Service: New method
1.17.3¶
- release-date
2018-09-18 4:00 P.M PDT
- release-by
Ask Solem (@ask)
Service: New attribute
mundane_level
decides the logging level of mundane logging events such as “[X] Starting…”, for starting/stopping and tasks being cancelled.The value for this must be a logger level name, and is
"info"
by default.If logging for a service is noisy at info-level, you can move it to debug level by setting this attribute to
"debug"
:class X(Service): mundane_level = 'debug'
1.17.2¶
- release-date
2018-09-17 3:00 P.M PDT
- release-by
Ask Solem (@ask)
Removed and fixed import from
collections
that will be moved tocollections.abc
in Python 3.8.This also silences a
DeprecationWarning
that was being emitted on Python 3.7.Type annotations now passing checks on mypy 0.630.
1.17.1¶
- release-date
2018-09-13 6:27 P.M PDT
- release-by
Ask Solem (@ask)
Fixes several bugs related to unwrapping
Optional[List[..]]
inmode.utils.objects.annotations()
.This functionality is not really related to mode at all, so should be moved out of this library. Faust uses it for models.
1.17.0¶
- release-date
2018-09-12 5:39 P.M PDT
- release-by
Ask Solem (@ask)
New async iterator utility:
arange
Like
range
but returns an async iterator:async for n in arange(0, 10, 2): ...
New async iterator utility:
aslice()
Like
itertools.islice
but works on asynchronous iterators.New async iterator utility:
chunks()
chunks
takes an async iterable and divides it up into chunks of size n:# Split range of 100 numbers into chunks of 10 each. async for chunk in chunks(arange(100), 10): yield chunk
This gives chunks like this:
[ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [10, 11, 12, 13, 14, 15, 16, 17, 18, 19], ..., ]
1.16.0¶
- release-date
2018-09-11 1:37 P.M PDT
- release-by
Ask Solem
Requirements
Now depends on Mode 1.15.1.
Contributed by Michael Seifert
- Distribution: Installing mode no longer installs the
t
directory containing tests as a Python package.
Contributed by Michael Seifert
- Distribution: Installing mode no longer installs the
Testing: New
AsyncContextManagerMock
You can use this to mock asynchronous context managers.
Please see
AsyncContextManagerMock
for an example.CI: Python 3.7.0 and 3.6.0 was added to the build matrix.
1.15.1¶
- release-date
2018-08-15 11:17 A.M PDT
- release-by
Ask Solem
Tests now passing on CPython 3.7.0
Utils: Adds
remove_optional
function inmode.utils.objects
This can be used to extract the concrete type from
Optional[Foo]
.Utils: Adds
humanize_seconds
function tomode.utils.times
1.15.0¶
- release-date
2018-06-27 1:39 P.M PDT
- release-by
Ask Solem
Worker: Logging can now be set up using dictionary config, by passing the
logging_config
argument tomode.Worker
.Contributed by Allison Wang.
Worker: No longer supports the
logformat
argument.To set up custom log format you must now pass in dict configuration via the
logging_config
argument.Service:
start()
accidentally silencedasyncio.CancelledError
.Service: Invalid assert caused
CrashingSupervisor
to crash with strange error
1.14.1¶
- release-date
2018-06-06 1:26 P.M PDT
- release-by
Ask Solem
Service: Fixed “coroutine x was never awaited” for background tasks (
@Service.task
decorator) when service is started and stopped in quick succession.
1.14.0¶
- release-date
2018-06-05 12:13 P.M PDT
- release-by
Ask Solem
Adds method
Service.wait_many(futures, *, timeout=None)
1.13.0¶
- release-date
2018-05-16 1:26 P.M PDT
- release-by
Ask Solem
Mode now registers as a library having static type annotations.
This conforms to PEP 561 – a new specification that defines how Python libraries register type stubs to make them available for use with static analyzers like mypy and pyre-check.
The code base now passes
--strict-optional
type checks.
1.12.5¶
- release-date
2018-05-14 4:48 P.M PDT
- release-by
Ask Solem
Supervisor: Fixed wrong index calculation in management of index-based service restart.
1.12.4¶
- release-date
2018-05-07 3:20 P.M PDT
- release-by
Ask Solem
Adds new mock class for async functions:
mode.utils.mocks.AsyncMock()
This can be used to mock an async callable:
from mode.utils.mocks import AsyncMock class App(Service): async def on_start(self): self.ret = await self.some_async_method('arg') async def some_async_method(self, arg): await asyncio.sleep(1) @pytest.fixture def app(): return App() @pytest.mark.asyncio async def test_something(*, app): app.some_async_method = AsyncMock() async with app: # starts and stops the service, calling on_start app.some_async_method.assert_called_once_with('arg') assert app.ret is app.some_async_method.coro.return_value
Added 100% test coverage for modules:
1.12.3¶
- release-date
2018-05-07 3:33 P.M PDT
- release-by
Ask Solem
Important Notes¶
Moved to https://github.com/ask/mode
Changes¶
Signal: Improved repr when signal has a default sender.
DictAttribute: Now supports
len
anddel(d[key])
.Worker: If overriding
on_first_start
you can now calldefault_on_first_start
instead of super.Example:
class MyWorker(Worker): async def on_first_start(self) -> None: print('FIRST START') await self.default_on_first_start()
1.12.2¶
- release-date
2018-04-26 11:47 P.M PDT
- release-by
Ask Solem
Fixed shutdown error in
ServiceThread
.
1.12.1¶
- release-date
2018-04-24 11:28 P.M PDT
- release-by
Ask Solem
Now works with CPython 3.6.1 and 3.6.0.
1.12.0¶
- release-date
2018-04-23 1:28 P.M PDT
- release-by
Ask Solem
Backward Incompatible Changes¶
Changed
Service.add_context
To add an async context manager (
AsyncContextManager
), useadd_async_context()
:class S(Service): async def on_start(self) -> None: self.context = await self.add_async_context(MyAsyncContext())
To add a regular context manager (
ContextManager
), useadd_context()
:class S(Service): async def on_start(self) -> None: self.context = self.add_context(MyContext())
This change was made so that contexts can be added from non-async functions. To add an async context you still need to be within an async function definition.
News¶
- Worker: Now redirects
sys.stdout
andsys.stderr
to the logging subsystem by default.
To disable this pass
Worker(redirect_stdouts=False)
.The default severity level for
print
statements arelogging.WARN
, but you can change this usingWorker(redirect_stdouts_level='INFO')
.
- Worker: Now redirects
Seconds
/want_seconds()
can now be expressed as strings and rate strings:float as string:
want_seconds('1.203') == 1.203
10 in one second:
want_seconds('10/s') == 10.0
10.33 in one hour:
want_seconds('10.3/h') == 0.0028611111111111116
100 in one hour:
want_seconds('100/h') == 0.02777777777777778
100 in one day:
want_seconds('100/d') == 0.0011574074074074076
This is especially useful for the rate argument to the
rate_limit
helper.Added new context manager:
mode.utils.logging.redirect_stdouts()
.Module
mode.types
now organized by category:Service types:
mode.types.services
Signal types:
mode.types.signals
Supervisor types:
mode.types.supervisors
mode.flight_recorder
can now wrap objects so that every method call on that object will result in the call and arguments to that call being logged.Example logging statements with
INFO
severity:with flight_recorder(logger, timeout=10.0) as on_timeout: redis = on_timeout.wrap_info(self.redis) await redis.get(key)
There’s also
wrap_debug(o)
,wrap_warn(o)
,wrap_error(o)
, and for any severity:wrap(logging.CRIT, o)
.
Fixes¶
Fixed bug in
Service.wait
on Python 3.7.
1.11.5¶
- release-date
2018-04-19 3:12 P.M PST
- release-by
Ask Solem
FlowControlQueue now available in
mode.utils.queues
.This is a backward compatible change.
Tests for FlowControlQueue
1.11.4¶
- release-date
2018-04-19 9:36 A.M PST
- release-by
Ask Solem
Adds
mode.flight_recorder
This is a logging utility to log stuff only when something times out.
For example if you have a background thread that is sometimes hanging:
class RedisCache(mode.Service): @mode.timer(1.0) def _background_refresh(self) -> None: self._users = await self.redis_client.get(USER_KEY) self._posts = await self.redis_client.get(POSTS_KEY)
You want to figure out on what line this is hanging, but logging all the time will provide way too much output, and will even change how fast the program runs and that can mask race conditions, so that they never happen.
Use the flight recorder to save the logs and only log when it times out:
logger = mode.get_logger(__name__) class RedisCache(mode.Service): @mode.timer(1.0) def _background_refresh(self) -> None: with mode.flight_recorder(logger, timeout=10.0) as on_timeout: on_timeout.info(f'+redis_client.get({USER_KEY!r})') await self.redis_client.get(USER_KEY) on_timeout.info(f'-redis_client.get({USER_KEY!r})') on_timeout.info(f'+redis_client.get({POSTS_KEY!r})') await self.redis_client.get(POSTS_KEY) on_timeout.info(f'-redis_client.get({POSTS_KEY!r})')
If the body of this
with
statement completes before the timeout, the logs are forgotten about and never emitted – if it takes more than ten seconds to complete, we will see these messages in the log:[2018-04-19 09:43:55,877: WARNING]: Warning: Task timed out! [2018-04-19 09:43:55,878: WARNING]: Please make sure it is hanging before restarting. [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (started at Thu Apr 19 09:43:45 2018) Replaying logs... [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:45 2018) +redis_client.get('user') [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:49 2018) -redis_client.get('user') [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:46 2018) +redis_client.get('posts') [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] -End of log-
Now we know this
redis_client.get
call can take too long to complete, and should consider adding a timeout to it.
1.11.3¶
- release-date
2018-04-18 5:22 P.M PST
- relese-by
Ask Solem
Cry handler (kill -USR1): Truncate huge data in stack frames.
ServiceProxy: Now supports
_crash
method.
1.11.2¶
- release-date
2018-04-18 5:02 P.M PST
- release-by
Ask Solem
Service:
add_future()
now maintains futures in a set and futures are automatically removed from it when done.Cry handler (kill -USR1) now shows name of Service.task background tasks.
Stampede: Now propagates cancellation.
1.11.1¶
- release-date
2018-04-18 11:08 P.M PST
- release-by
Ask Solem
Service.add_context: Now works with AsyncContextManager.
CI now runs functional tests.
Added supervisor and service tests.
1.11.0¶
- release-date
2018-04-17 1:23 P.M PST
- release-by
Ask Solem
Supervisor: Fixes bug with max restart triggering too early.
Supervisor: Also restart child services.
Service: Now supports
__post_init__
like Python 3.7 dataclasses.Service: Crash is logged even if crashed multiple times.
1.10.4¶
- release-date
2018-04-13 3:53 P.M PST
- release-by
Ask Solem
Supervisor: Log full traceback when restarting service.
1.10.3¶
- release-date
2018-04-11 10:58 P.M PST
- release-by
Ask Solem
setup_logging: now ensure logging is setup by clearing root logger handlers.
1.10.2¶
- release-date
2018-04-03 4:50 P.M PST
- release-by
Ask Solem
Fixed wrong version number in Changelog.
1.10.1¶
- release-date
2018-04-03 4:43 P.M PST
- release-by
Ask Solem
- Service.wait: If the future we are waiting for is cancelled we must
propagate
CancelledError
.
1.10.0¶
- release-date
2018-03-30 12:36 P.M PST
- release-by
Ask Solem
New supervisor:
ForfeitOneForOneSupervisor
.If a service in the group crashes we give up on that service and don’t start it again.
New supervisor:
ForfeitOneForAllSupervisor
.If a service in the group crashes we give up on it, but also stop all services in the group and give up on them also.
Service Logging: Renamed
self.log.crit
toself.log.critical
.The old name is still available and is not deprecated at this time.
1.9.2¶
- release-date
2018-03-20 10:17 P.M PST
- release-by
Ask Solem
Adds
FlowControlEvent.clear()
to clear all contents of flow controlled queues.FlowControlEvent
now starts in a suspended state.To disable this pass
FlowControlEvent(initially_suspended=False))
.Adds
Service.service_reset
method to reset service start/stopped/crashed/etc., flags
1.9.1¶
- release-date
2018-03-05 11:51 P.M PST
- release-by
Ask Solem
No longer depends on terminaltables.
1.9.0¶
- release-date
2018-03-05 11:33 P.M PST
- release-by
Ask Solem
Backward Incompatible Changes¶
Module
mode.utils.debug
renamed tomode.debug
.This is unlikely to affect users as this module is only used by mode internally.
This module had to move because it imports
mode.Service
, and themode.utils
package is not allowed to import from themode
package at all.
News¶
Added function
mode.utils.import.smart_import()
.Added non-async version of
mode.Signal
:mode.SyncSignal
.The signal works exactly the same as the asynchronous version, except
Signal.send
must not beawait
-ed:on_configured = SyncSignal() on_configured.send(sender=obj)
Added method
iterate
tomode.utils.imports.FactoryMapping
.This enables you to iterate over the extensions added to a
setuptools
entrypoint.
Fixes¶
StampedeWrapper
now correctly clears flag when original call done.
1.8.0¶
- release-date
2018-02-20 04:01 P.M PST
- release-by
Ask Solem
Backward Incompatible Changes¶
API Change to fix memory leak in
Service.wait
.The
Service.wait(*futures)
method was added to be able to wait for this list of futures but also stop waiting if the service is stopped or crashed:import asyncio from mode import Service class X(Service): on_thing_ready: asyncio.Event def __post_init__(self): self.on_thing_ready = asyncio.Event(loop=loop) @Service.task async def _my_background_task(self): while not self.should_stop: # wait for flag to be set (or service stopped/crashed) await self.wait(self.on_thing_ready.wait()) print('FLAG SET')
The problem with this was
The wait flag would return None and not raise an exception if the service is stopped/crashed.
Futures would be scheduled on the event loop but not properly cleaned up, creating a very slow memory leak.
No return value was returned for succesful feature.
So to properly implement this we had to change the API of the
wait
method to return a tuple instead, and to only allow a single coroutine to be passed to wait:@Service.task async def _my_background_task(self): while not self.should_stop: # wait for flag to be set (or service stopped/crashed) result, stopped = await self.wait(self.on_thing_ready) if not stopped: print('FLAG SET')
This way the user can provide an alternate path when the service is stopped/crashed while waiting for this event.
A new shortcut method
wait_for_stopped(fut)
was also added:# wait for flag to be set (or service stopped/crashed) if not await self.wait_for_stopped(self.on_thing_ready): print('FLAG SET')
Moreover, you can now pass
asyncio.Event
objects directly towait()
.
News¶
Bugs¶
Signals can create clone of signal with default sender already set
signal: Signal[int] = Signal() signal = signal.with_default_sender(obj)
1.7.0¶
- release-date
2018-02-05 12:28 P.M PST
- release-by
Ask Solem
Adds
mode.utils.aiter
for missingaiter
andanext
functions.Adds
mode.utils.futures
forasyncio.Task
related tools.Adds
mode.utils.collections
for custom mapping/set and list data structures.Adds
mode.utils.imports
for importing modules at runtime, as well as utilities for typedsetuptools
entry-points.Adds
mode.utils.text
for fuzzy matching user input.
1.6.0¶
- release-date
2018-02-05 11:10 P.M PST
- release-by
Ask Solem
Fixed bug where
@Service.task
background tasks were not started in subclasses.Service: Now has two exit stacks:
.exit_stack
&.async_exit_stack
.This is a backward incompatible change, but probably nobody was accessing
.exit_stack
directly.Use
await Service.enter_context(ctx)
with both regular and asynchronous context managers:class X(Service): async def on_start(self) -> None: # works with both context manager types. await self.enter_context(async_context) await self.enter_context(context)
Adds
asynccontextmanager`()
decorator from CPython 3.7b1.This decorator works exactly the same as
contextlib.contextmanager()
, but forasync with
.Import it from
mode.utils.contexts
:from mode.utils.contexts import asynccontextmanager @asynccontextmanager async def connection_or_default(conn: Connection = None) -> Connection: if connection is None: async with connection_pool.acquire(): yield else: yield connection async def main(): async with connection_or_default() as connection: ...
Adds
AsyncExitStack
from CPython 3.7b1This works like
contextlib.ExitStack
, but for asynchronous context managers used withasync with
.Logging: Worker debug log messages are now colored blue when colors are enabled.
1.5.0¶
- release-date
2018-01-04 03:43 P.M PST
- release-by
Ask Solem
Service: Adds new
await self.add_context(context)
This adds a new context manager to be entered when the service starts, and exited once the service exits.
The context manager can be either a
typing.AsyncContextManager
(async with
) or a regulartyping.ContextManager
(with
).Service: Added
await self.add_runtime_dependency()
which unlikeadd_dependency
starts the dependent service if the self is already started.Worker: Now supports a new
console_port
argument to specify a port for the aiomonitor console, different than the default (50101).Note
The aiomonitor console is only started when
Worker(debug=True, ...)
is set.
1.4.0¶
- release-date
2017-12-21 09:50 A.M PST
- release-by
Ask Solem
Worker: Add support for parameterized logging handlers.
Contributed by Prithvi Narasimhan.
1.3.0¶
- release-date
2017-12-04 01:17 P.M PST
- release-by
Ask Solem
Now supports color output in logs when logging to a terminal.
Now depends on colorlog
Added
mode.Signal
: async. implementation of the observer pattern (think Django signals).DependencyGraph is now a generic type:
DependencyGraph[int]
Node is now a generic type:
Node[Service]
.
1.2.1¶
- release-date
2017-11-06 04:50 P.M PST
- release-by
Ask Solem
Service: Subclasses can now override a Service.task method.
Previously it would unexpectedly start two tasks: the task defined in the superclass and the task defined in the subclass.
1.2.0¶
- release-date
2017-11-02 03:17 P.M PDT
- release-by
Ask Solem
Renames PoisonpillSupervisor to CrashingSupervisor.
Child services now stopped even if not fully started.
Previously
child_service.stop()
would not be called if child_service.start() never completed, but as a service might be in the process of starting other child services, we need to call stop even if not fully started.
1.1.1¶
- release-date
2017-10-25 04:34 P.M PDT
- release-by
Ask Solem
Added alternative event loop implementations: eventlet, gevent, uvloop.
E.g. to use gevent as the event loop, install mode using:
$ pip install mode[gevent]
and add this line to the top of your worker entrypoint module:
import mode.loop mode.loop.use('gevent')
Service: More fixes for the weird __init_subclass__ behavior only seen in Python 3.6.3.
ServiceThread: Now propagates errors raised in the thread to the main thread.
1.1.0¶
- release-date
2017-10-19 01:35 P.M PDT
- release-by
Ask Solem
ServiceThread: Now inherits from Service, and uses
loop.run_in_executor()
to start the service as a thread.setup_logging: filename argument is now respected.
1.0.2¶
- release-date
2017-10-10 01:51 P.M PDT
- release-by
Ask Solem
Adds support for Python 3.6.0
Adds backports of typing improvements in CPython 3.6.1 to
mode.utils.compat
:AsyncContextManager
,ChainMap
,Counter
, andDeque
.Supervisor.add
and.discard
now takes an arbitrary number of services to add/discard as star arguments.Fixed typo in example:
Service.task
->mode.Service.task
.Contributed by Xu Jing.
1.0.1¶
- release-date
2017-10-05 02:53 P.M PDT
- release-by
Ask Solem
Fixes compatibility with Python 3.6.3.
Python 3.6.3 badly broke
__init_subclass__
, in such a way that any class attribute set is set for all subclasses.