Change history

4.0.1

release-date

2019-07-17 3:42 P.M PST

release-by

Ask Solem (@ask)

  • Utils: stampede objects can now be read by Sphinx and inspect.signature().

  • CI: Adds CPython 3.7.3 to build matrix, and set as default for lint stages

4.0.0

release-date

2019-05-07 2:21 P.M PST

release-by

Ask Solem (@ask)

  • 100% Test Coverage

  • Fixed several edge case bugs that were never reported.

3.2.2

release-date

2019-04-07 6:18 P.M PST

release-by

Ask Solem (@ask)

3.2.1

release-date

2019-04-07 4:07 P.M PST

release-by

Ask Solem (@ask)

3.2.0

release-date

2019-04-06 11:00 P.M PST

release-by

Ask Solem (@ask)

  • Adds Service.itertimer: used to perform periodic tasks, but with automatic drift adjustment.

  • Adds mode.utils.mocks.ContextMock()

    To mock a regular context manager.

3.1.3

release-date

2019-04-04 08:41 P.M PST

release-by

Ask Solem (@ask)

  • mode.utils.worker.exiting now takes option to print exceptions.

  • Threads: Method queue “starting…” logs now logged with debug severity.

  • Worker: execute_from_commandline no longer swallow errors if loop closed.

  • Adds mode.locals.LocalStack.

3.1.2

release-date

2019-04-04 08:37 P.M PST

release-by

Ask Solem (@ask)

  • Revoked release: Version without changelog entry was uploaded to PyPI. Please upgrade to 3.1.3.

3.1.1

release-date

2019-03-27 10:02 A.M PST

release-by

Ask Solem (@ask)

  • Service: property should_stop is now true if service crashed.

  • Timers: Avoid drift + introduce a tiny bit of drift to timers.

    Thanks to Bob Haddleton (@bobh66) for discovering this issue.

3.1.0

release-date

2019-03-21 03:26 P.M PST

release-by

Ask Solem (@ask)

3.0.13

release-date

2019-03-20 04:58 P.M PST

release-by

Ask Solem (@ask)

  • Adds CompositeLogger.warning alias to warn.

    flake8-logging-format has a rule that says you are only allowed to use .warning, so going with that.

3.0.12

release-date

2019-03-20 03:23 P.M PST

release-by

Ask Solem (@ask)

  • Adds all_tasks() as a backward compatible asyncio.all_tasks().

  • Signal: Fixes .connect() decorator to work with parens and without

    Signal decorator now works with parens:

    @signal.connect()
    def my_handler(sender, **kwargs):
        ...
    

    and without parens:

    @signal.connect
    def my_handler(sender, **kwargs):
    ...
    
  • Signal: Do not use weakref by default.

    Using weakref by default meant it was too easy to connect a signal handler to only have it disappear because there were no more references to the object.

3.0.11

release-date

2019-03-19 08:50 A.M PST

release-by

Ask Solem (@ask)

  • Adds ThrowableQueue._throw() for non-async version of .throw().

3.0.10

release-date

2019-03-14 03:55 P.M PST

release-by

Ask Solem (@ask)

  • Worker: was giving successful exit code when an exception is raised.

    The .execute_from_commandline method now always exits (its return type is typing.NoReturn).

  • Adds NoReturn to mode.utils.compat.

    Import typing.NoReturn from here to support Python versions before 3.6.3.

3.0.9

release-date

2019-03-08 01:20 P.M PDT

release-by

Ask Solem (@ask)

  • Threads: Add multiple workers for thread method queue.

    Default number of workers is now 2, to allow for two recursive calls.

  • Signal: Use Signal.label instead of .indent to be more consistent.

  • Signal: Properly set .name when signal is member of class.

  • Adds ability to log the FULL traceback of asyncio.Task.

  • Service: Stop faster if stopped immediately after start

  • Service: Correctly track dependencies for services added using Service.on_init_dependencies (Issue #40).

    Contributed by Nimi Wariboko Jr (@nemosupremo).

3.0.8

release-date

2019-01-25 03:54 P.M PDT

release-by

Ask Solem (@ask)

  • Fixes DeprecationWarning importing from collections.

  • stampede: Fixed edge case where stampede wrapped function called multiple times.

    Calling the same stampede wrapped function multiple times within the same event loop iteration would previously call the function multiple times.

    For example using asyncio.gather():

    from mode.utils.futures import stampede
    
    count = 0
    @stampede
    async def update_count():
        global count
        count += 1
    
    async def main():
        await asyncio.gather(
            update_count(),
            update_count(),
            update_count(),
            update_count(),
        )
    
        assert count == 1
    

    Previously this would call the function four times, but with the fix it’s only called once and provides the expected result.

  • Mocks: Adds mask_module() and

    patch_module().

  • CI: Added Windows build.

  • CI: Enabled random order for tests.

3.0.7

release-date

2019-01-18 01:12 P.M PDT

release-by

Ask Solem (@ask)

  • ServiceThread .stop() would wait for thread shutdown even if thread was never started.

  • CI: Adds CPython 3.7.2 and 3.6.8 to build matrix

3.0.6

release-date

2019-01-07 12:10 P.M PDT

release-by

Ask Solem (@ask)

  • Adds %(extra)s as log format option.

    To add additional context to your logging statements use for example:

    logger.error('Foo', extra={'data': {'database': 'db1'}})
    

3.0.5

release-date

2018-12-19 04:40 P.M PDT

release-by

Ask Solem (@ask)

3.0.4

release-date

2018-12-07 04:40 P.M PDT

release-by

Ask Solem (@ask)

3.0.3

release-date

2018-12-07 3:22 P.M PDT

release-by

Ask Solem (@ask)

  • Threads: Fixed delay in shutdown if on_thread_stop callback raises exception.

  • Service: Stopping of children no longer propagates exceptions, to ensure other services are still stopped.

  • Worker: Fixed race condition if worker stopped before being fully started.

    This would lead the worker to shutdown early before fully stopping all dependent services.

  • Tests: Adds AsyncMagicMock

3.0.2

release-date

2018-12-07 1:14 P.M PDT

release-by

Ask Solem (@ask)

  • Worker: Fixes crash on Windows where signal handlers cannot be registered.

  • Utils: Adds shortname() to get non-qualified object path.

  • Utils: Adds canonshortname() to get non-qualified object path that attempts to resolve the real name of __main__.

3.0.1

release-date

2018-12-06 10:20 A.M PDT

release-by

Ask Solem (@ask)

  • Worker: Added new callback on_worker_shutdown.

  • Worker: Do not stop twice, instead wait for original stop to complete.

    Signals would start multiple stopping coroutines, leading to the worker shutting down too fast.

  • Threads: All ServiceThread services needs a keepalive coroutine to be scheduled.

  • Supervisor: Fixed issue with CrashingSupervisor where service would not crash.

3.0.0

release-date

2018-11-30 4:48 P.M PDT

release-by

Ask Solem (@ask)

  • ServiceThread no longer uses run_in_executor.

    Since services are long running, it is not a good idea for them to block pool worker threads. Instead we run one thread for every ServiceThread.

  • Adds QueuedServiceThread

    This subclass of ServiceThread enables the use of a queue to send work to the service thread.

    This is useful for services that wrap blocking network clients for example.

    If you have a blocking Redis client you could run it in a separate thread like this:

    class Redis(QueuedServiceThread):
        _client: StrictRedis = None
    
        async def on_start(self) -> None:
            self._client = StrictRedis()
    
        async def get(self, key):
            return await self.call_thread(self._client.get, key)
    
        async def set(self, key, value):
            await self.call_thread(self._client.set, key, value)
    

    The actual redis client will be running in a separate thread (with a separate event loop). The get and set methods will delegate to the thread, and return only when the thread is finished handling them and is ready with a result:

    async def use_redis():
        # We use async-with-statement here, but
        # can also do `await redis.start()` then `await redis.stop()`
        async with Redis() as redis:
            await redis.set(key='foo', value='bar')
            assert await redis.get(key='foo') == 'bar'
    
  • Collections: FastUserSet and ManagedUserSet now implements all set operations.

  • Collections are now generic types.

    You can now subclass collections with typing information:

    • class X(FastUserDict[str, int]): ...

    • class X(ManagedUserDict[str, int]): ...

    • class X(FastUserSet[str]): ...

    • class X(ManagedUserSet[str]): ...

  • maybe_async() utility now also works with @asyncio.coroutine decorated coroutines.

  • Worker: SIGUSR1 cry handler: Fixed crash when coroutine does not have __name__ attribute.

2.0.4

release-date

2018-11-19 1:07 P.M PST

release-by

Ask Solem (@ask)

  • FlowControlQueue.clear now cancels all waiting for Queue.put.

2.0.3

release-date

2018-11-05 5:20 P.M PDT

release-by

Ask Solem (@ask)

  • Adds Service.wait_first(*coros)

    Wait for the first coroutine to return, where coroutines can also be asyncio.Event.

    Returns mode.services.WaitResults with fields:

    • .done - List of arguments that are now done.

    • .results - List of return values in order of .done.

    • .stopped - Set to True if the service was stopped.

2.0.2

release-date

2018-11-03 9:07 A.M PST

release-by

Ask Solem (@ask)

  • Now depends on aiocontextvars 0.2

    This release uses PEP 508 syntax for conditional requirements, as 2.0.1 did not work when installing wheel.

2.0.1

release-date

2018-11-02 7:38 P.M PST

release-by

Ask Solem (@ask)

2.0.0

release-date

2018-11-02 9:12 A.M PST

release-by

Ask Solem (@ask)

  • Services now create the event loop on demand.

    This means the event loop is no longer created in Service.__init__ so that services can be defined at module scope without initializing the loop.

    This makes the ServiceProxy pattern redundant for most use cases.

  • Adds .utils.compat.current_task as alias for asyncio.current_task.

  • Adds support for contextvars in Python 3.6 using aiocontextvars.

    In mode services you can now use contextvars module even on Python 3.6, thanks to the work of @fantix.

1.18.2

release-date

2018-11-30 6:23 P.M PDT

release-by

Ask Solem (@ask)

  • Worker: SIGUSR1 cry handler: Fixed crash when coroutine does not have __name__ attribute.

1.18.1

release-date

2018-10-03 2:49 P.M PDT

release-by

Ask Solem (@ask)

  • Service: Service.from_awaitable(coro) improvements.

    The resulting service.start will now:

    • Convert awaitable to asyncio.Task.

    • Wait for task to complete.

    then service.stop will:

    • Cancel the task.

    This ensures an asyncio.sleep(10.0) within can be cancelled. If you need some operation to absolutely finish you must use asyncio.shield.

  • Utils: cached_property adds new .is_set(o) method on descriptor

    This can be used to test for the attribute having been cached/used.

    If you have a class with a cached_property:

    from mode.utils.objects import cached_property
    
    class X:
    
        @cached_property
        def foo(self):
            return 42
    
    x = X()
    print(x.foo)
    

    From an instance you can now check if the property was accessed:

    if type(x).foo.is_set(x):
        print(f'Someone accessed x.foo and it was cached as: {x.foo}')
    

1.18.0

release-date

2018-10-02 3:32 P.M PDT

release-by

Ask Solem (@ask)

  • Worker: Fixed error when starting aioconsole on --debug

    The worker would crash with:

    TypeError: Use `self.add_context(ctx)` for non-async context
    

    when started with the --debug flag.

  • Worker: New daemon argument controls shutdown of worker.

    When the flag is enabled, the default, the worker will not shut down until the worker instance is either explicitly stopped, or it receives a terminating process signal (SIGINT/SIGTERM/etc.)

    When disabled, the worker for the given service will shut down as soon as await service.start() returns.

    You can think of it as a flag for daemons, but one that doesn’t actually do any of the UNIX daemonization stuff (detaching, etc.). It merely means the worker continues to run in the background until stopped by signal.

  • Service: Added class method: Service.from_awaitable.

    This can be used to create a service out of any coroutine or Awaitable:

    from mode import Service, Worker
    
    async def me(interval=1.0):
        print('ME STARTING')
        await asyncio.sleep(interval)
        print('ME STOPPING')
    
    def run_worker(interval=1.0):
        coro = me(interval=1.0)
        Worker(Service.from_awaitable(coro)).execute_from_commandline()
    
    if __name__ == '__main__':
        run_worker()
    

    Note

    Using a service with await self.sleep(1.0) is often not what you want, as stopping the service will have to wait for the sleep to finish.

    Service.from_awaitable is as such a last resort for cases where you’re provided a coroutine you cannot implement as a service.

    Service.sleep() is useful as it will stop sleeping immediately if the service is stopped:

    class Me(Service):

    async def on_start(self) -> None:

    await self.sleep(1.0)

  • Service: New method _repr_name can be used to override the service

    class name used in repr(service).

1.17.3

release-date

2018-09-18 4:00 P.M PDT

release-by

Ask Solem (@ask)

  • Service: New attribute mundane_level decides the logging level of mundane logging events such as “[X] Starting…”, for starting/stopping and tasks being cancelled.

    The value for this must be a logger level name, and is "info" by default.

    If logging for a service is noisy at info-level, you can move it to debug level by setting this attribute to "debug":

    class X(Service):
        mundane_level = 'debug'
    

1.17.2

release-date

2018-09-17 3:00 P.M PDT

release-by

Ask Solem (@ask)

  • Removed and fixed import from collections that will be moved to collections.abc in Python 3.8.

    This also silences a DeprecationWarning that was being emitted on Python 3.7.

  • Type annotations now passing checks on mypy 0.630.

1.17.1

release-date

2018-09-13 6:27 P.M PDT

release-by

Ask Solem (@ask)

  • Fixes several bugs related to unwrapping Optional[List[..]] in mode.utils.objects.annotations().

    This functionality is not really related to mode at all, so should be moved out of this library. Faust uses it for models.

1.17.0

release-date

2018-09-12 5:39 P.M PDT

release-by

Ask Solem (@ask)

  • New async iterator utility: arange

    Like range but returns an async iterator:

    async for n in arange(0, 10, 2):
        ...
    
  • New async iterator utility: aslice()

    Like itertools.islice but works on asynchronous iterators.

  • New async iterator utility: chunks()

    chunks takes an async iterable and divides it up into chunks of size n:

    # Split range of 100 numbers into chunks of 10 each.
    async for chunk in chunks(arange(100), 10):
        yield chunk
    

    This gives chunks like this:

    [
        [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
        [10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
        ...,
    ]
    

1.16.0

release-date

2018-09-11 1:37 P.M PDT

release-by

Ask Solem

  • Requirements

    • Now depends on Mode 1.15.1.

      Contributed by Michael Seifert

  • Distribution: Installing mode no longer installs the t directory

    containing tests as a Python package.

    Contributed by Michael Seifert

  • Testing: New AsyncContextManagerMock

    You can use this to mock asynchronous context managers.

    Please see AsyncContextManagerMock for an example.

  • CI: Python 3.7.0 and 3.6.0 was added to the build matrix.

1.15.1

release-date

2018-08-15 11:17 A.M PDT

release-by

Ask Solem

  • Tests now passing on CPython 3.7.0

  • Utils: Adds remove_optional function in mode.utils.objects

    This can be used to extract the concrete type from Optional[Foo].

  • Utils: Adds humanize_seconds function to mode.utils.times

1.15.0

release-date

2018-06-27 1:39 P.M PDT

release-by

Ask Solem

  • Worker: Logging can now be set up using dictionary config, by passing the logging_config argument to mode.Worker.

    Contributed by Allison Wang.

  • Worker: No longer supports the logformat argument.

    To set up custom log format you must now pass in dict configuration via the logging_config argument.

  • Service: start() accidentally silenced asyncio.CancelledError.

  • Service: Invalid assert caused CrashingSupervisor to crash with strange error

1.14.1

release-date

2018-06-06 1:26 P.M PDT

release-by

Ask Solem

  • Service: Fixed “coroutine x was never awaited” for background tasks (@Service.task decorator) when service is started and stopped in quick succession.

1.14.0

release-date

2018-06-05 12:13 P.M PDT

release-by

Ask Solem

  • Adds method Service.wait_many(futures, *, timeout=None)

1.13.0

release-date

2018-05-16 1:26 P.M PDT

release-by

Ask Solem

  • Mode now registers as a library having static type annotations.

    This conforms to PEP 561 – a new specification that defines how Python libraries register type stubs to make them available for use with static analyzers like mypy and pyre-check.

  • The code base now passes --strict-optional type checks.

1.12.5

release-date

2018-05-14 4:48 P.M PDT

release-by

Ask Solem

  • Supervisor: Fixed wrong index calculation in management of index-based service restart.

1.12.4

release-date

2018-05-07 3:20 P.M PDT

release-by

Ask Solem

  • Adds new mock class for async functions: mode.utils.mocks.AsyncMock()

    This can be used to mock an async callable:

    from mode.utils.mocks import AsyncMock
    
    class App(Service):
    
        async def on_start(self):
            self.ret = await self.some_async_method('arg')
    
        async def some_async_method(self, arg):
            await asyncio.sleep(1)
    
    @pytest.fixture
    def app():
        return App()
    
    @pytest.mark.asyncio
    async def test_something(*, app):
        app.some_async_method = AsyncMock()
        async with app: # starts and stops the service, calling on_start
            app.some_async_method.assert_called_once_with('arg')
            assert app.ret is app.some_async_method.coro.return_value
    
  • Added 100% test coverage for modules:

1.12.3

release-date

2018-05-07 3:33 P.M PDT

release-by

Ask Solem

Important Notes

Changes

  • Signal: Improved repr when signal has a default sender.

  • DictAttribute: Now supports len and del(d[key]).

  • Worker: If overriding on_first_start you can now call default_on_first_start instead of super.

    Example:

    class MyWorker(Worker):
    
        async def on_first_start(self) -> None:
            print('FIRST START')
            await self.default_on_first_start()
    

1.12.2

release-date

2018-04-26 11:47 P.M PDT

release-by

Ask Solem

1.12.1

release-date

2018-04-24 11:28 P.M PDT

release-by

Ask Solem

  • Now works with CPython 3.6.1 and 3.6.0.

1.12.0

release-date

2018-04-23 1:28 P.M PDT

release-by

Ask Solem

Backward Incompatible Changes

  • Changed Service.add_context

    • To add an async context manager (AsyncContextManager), use add_async_context():

      class S(Service):
      
          async def on_start(self) -> None:
              self.context = await self.add_async_context(MyAsyncContext())
      
    • To add a regular context manager (ContextManager), use add_context():

      class S(Service):
      
          async def on_start(self) -> None:
              self.context = self.add_context(MyContext())
      

    This change was made so that contexts can be added from non-async functions. To add an async context you still need to be within an async function definition.

News

  • Worker: Now redirects sys.stdout and sys.stderr to the

    logging subsystem by default.

    • To disable this pass Worker(redirect_stdouts=False).

    • The default severity level for print statements are logging.WARN, but you can change this using Worker(redirect_stdouts_level='INFO').

  • Seconds/want_seconds() can now be expressed as strings and rate strings:

    • float as string: want_seconds('1.203') == 1.203

    • 10 in one second: want_seconds('10/s') == 10.0

    • 10.33 in one hour: want_seconds('10.3/h') == 0.0028611111111111116

    • 100 in one hour: want_seconds('100/h') == 0.02777777777777778

    • 100 in one day: want_seconds('100/d') == 0.0011574074074074076

    This is especially useful for the rate argument to the rate_limit helper.

  • Added new context manager: mode.utils.logging.redirect_stdouts().

  • Module mode.types now organized by category:

  • mode.flight_recorder can now wrap objects so that every method call on that object will result in the call and arguments to that call being logged.

    Example logging statements with INFO severity:

    with flight_recorder(logger, timeout=10.0) as on_timeout:
        redis = on_timeout.wrap_info(self.redis)
        await redis.get(key)
    

    There’s also wrap_debug(o), wrap_warn(o), wrap_error(o), and for any severity: wrap(logging.CRIT, o).

Fixes

  • Fixed bug in Service.wait on Python 3.7.

1.11.5

release-date

2018-04-19 3:12 P.M PST

release-by

Ask Solem

  • FlowControlQueue now available in mode.utils.queues.

    This is a backward compatible change.

  • Tests for FlowControlQueue

1.11.4

release-date

2018-04-19 9:36 A.M PST

release-by

Ask Solem

  • Adds mode.flight_recorder

    This is a logging utility to log stuff only when something times out.

    For example if you have a background thread that is sometimes hanging:

    class RedisCache(mode.Service):
    
        @mode.timer(1.0)
        def _background_refresh(self) -> None:
            self._users = await self.redis_client.get(USER_KEY)
            self._posts = await self.redis_client.get(POSTS_KEY)
    

    You want to figure out on what line this is hanging, but logging all the time will provide way too much output, and will even change how fast the program runs and that can mask race conditions, so that they never happen.

    Use the flight recorder to save the logs and only log when it times out:

    logger = mode.get_logger(__name__)
    
    class RedisCache(mode.Service):
    
        @mode.timer(1.0)
        def _background_refresh(self) -> None:
            with mode.flight_recorder(logger, timeout=10.0) as on_timeout:
                on_timeout.info(f'+redis_client.get({USER_KEY!r})')
                await self.redis_client.get(USER_KEY)
                on_timeout.info(f'-redis_client.get({USER_KEY!r})')
    
                on_timeout.info(f'+redis_client.get({POSTS_KEY!r})')
                await self.redis_client.get(POSTS_KEY)
                on_timeout.info(f'-redis_client.get({POSTS_KEY!r})')
    

    If the body of this with statement completes before the timeout, the logs are forgotten about and never emitted – if it takes more than ten seconds to complete, we will see these messages in the log:

    [2018-04-19 09:43:55,877: WARNING]: Warning: Task timed out!
    [2018-04-19 09:43:55,878: WARNING]: Please make sure it is hanging before restarting.
    [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (started at Thu Apr 19 09:43:45 2018) Replaying logs...
    [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:45 2018) +redis_client.get('user')
    [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:49 2018) -redis_client.get('user')
    [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:46 2018) +redis_client.get('posts')
    [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] -End of log-
    

    Now we know this redis_client.get call can take too long to complete, and should consider adding a timeout to it.

1.11.3

release-date

2018-04-18 5:22 P.M PST

relese-by

Ask Solem

  • Cry handler (kill -USR1): Truncate huge data in stack frames.

  • ServiceProxy: Now supports _crash method.

1.11.2

release-date

2018-04-18 5:02 P.M PST

release-by

Ask Solem

  • Service: add_future() now maintains futures in a set and futures are automatically removed from it when done.

  • Cry handler (kill -USR1) now shows name of Service.task background tasks.

  • Stampede: Now propagates cancellation.

1.11.1

release-date

2018-04-18 11:08 P.M PST

release-by

Ask Solem

  • Service.add_context: Now works with AsyncContextManager.

  • CI now runs functional tests.

  • Added supervisor and service tests.

1.11.0

release-date

2018-04-17 1:23 P.M PST

release-by

Ask Solem

  • Supervisor: Fixes bug with max restart triggering too early.

  • Supervisor: Also restart child services.

  • Service: Now supports __post_init__ like Python 3.7 dataclasses.

  • Service: Crash is logged even if crashed multiple times.

1.10.4

release-date

2018-04-13 3:53 P.M PST

release-by

Ask Solem

  • Supervisor: Log full traceback when restarting service.

1.10.3

release-date

2018-04-11 10:58 P.M PST

release-by

Ask Solem

  • setup_logging: now ensure logging is setup by clearing root logger handlers.

1.10.2

release-date

2018-04-03 4:50 P.M PST

release-by

Ask Solem

  • Fixed wrong version number in Changelog.

1.10.1

release-date

2018-04-03 4:43 P.M PST

release-by

Ask Solem

  • Service.wait: If the future we are waiting for is cancelled we must

    propagate CancelledError.

1.10.0

release-date

2018-03-30 12:36 P.M PST

release-by

Ask Solem

  • New supervisor: ForfeitOneForOneSupervisor.

    If a service in the group crashes we give up on that service and don’t start it again.

  • New supervisor: ForfeitOneForAllSupervisor.

    If a service in the group crashes we give up on it, but also stop all services in the group and give up on them also.

  • Service Logging: Renamed self.log.crit to self.log.critical.

    The old name is still available and is not deprecated at this time.

1.9.2

release-date

2018-03-20 10:17 P.M PST

release-by

Ask Solem

  • Adds FlowControlEvent.clear() to clear all contents of flow controlled queues.

  • FlowControlEvent now starts in a suspended state.

    To disable this pass FlowControlEvent(initially_suspended=False)).

  • Adds Service.service_reset method to reset service start/stopped/crashed/etc., flags

1.9.1

release-date

2018-03-05 11:51 P.M PST

release-by

Ask Solem

1.9.0

release-date

2018-03-05 11:33 P.M PST

release-by

Ask Solem

Backward Incompatible Changes

  • Module mode.utils.debug renamed to mode.debug.

    This is unlikely to affect users as this module is only used by mode internally.

    This module had to move because it imports mode.Service, and the mode.utils package is not allowed to import from the mode package at all.

News

  • Added function mode.utils.import.smart_import().

  • Added non-async version of mode.Signal: mode.SyncSignal.

    The signal works exactly the same as the asynchronous version, except Signal.send must not be await-ed:

    on_configured = SyncSignal()
    on_configured.send(sender=obj)
    
  • Added method iterate to mode.utils.imports.FactoryMapping.

    This enables you to iterate over the extensions added to a setuptools entrypoint.

Fixes

  • StampedeWrapper now correctly clears flag when original call done.

1.8.0

release-date

2018-02-20 04:01 P.M PST

release-by

Ask Solem

Backward Incompatible Changes

  • API Change to fix memory leak in Service.wait.

    The Service.wait(*futures) method was added to be able to wait for this list of futures but also stop waiting if the service is stopped or crashed:

    import asyncio
    from mode import Service
    
    class X(Service):
        on_thing_ready: asyncio.Event
    
        def __post_init__(self):
            self.on_thing_ready = asyncio.Event(loop=loop)
    
        @Service.task
        async def _my_background_task(self):
            while not self.should_stop:
                # wait for flag to be set (or service stopped/crashed)
                await self.wait(self.on_thing_ready.wait())
                print('FLAG SET')
    

    The problem with this was

    1. The wait flag would return None and not raise an exception if the service is stopped/crashed.

    2. Futures would be scheduled on the event loop but not properly cleaned up, creating a very slow memory leak.

    3. No return value was returned for succesful feature.

    So to properly implement this we had to change the API of the wait method to return a tuple instead, and to only allow a single coroutine to be passed to wait:

    @Service.task
    async def _my_background_task(self):
        while not self.should_stop:
            # wait for flag to be set (or service stopped/crashed)
            result, stopped = await self.wait(self.on_thing_ready)
            if not stopped:
                print('FLAG SET')
    

    This way the user can provide an alternate path when the service is stopped/crashed while waiting for this event.

    A new shortcut method wait_for_stopped(fut) was also added:

    # wait for flag to be set (or service stopped/crashed)
    if not await self.wait_for_stopped(self.on_thing_ready):
        print('FLAG SET')
    

    Moreover, you can now pass asyncio.Event objects directly to wait().

Bugs

  • Signals can create clone of signal with default sender already set

    signal: Signal[int] = Signal()
    signal = signal.with_default_sender(obj)
    

1.7.0

release-date

2018-02-05 12:28 P.M PST

release-by

Ask Solem

1.6.0

release-date

2018-02-05 11:10 P.M PST

release-by

Ask Solem

  • Fixed bug where @Service.task background tasks were not started in subclasses.

  • Service: Now has two exit stacks: .exit_stack & .async_exit_stack.

    This is a backward incompatible change, but probably nobody was accessing .exit_stack directly.

    Use await Service.enter_context(ctx) with both regular and asynchronous context managers:

    class X(Service):
    
        async def on_start(self) -> None:
            # works with both context manager types.
            await self.enter_context(async_context)
            await self.enter_context(context)
    
  • Adds asynccontextmanager`() decorator from CPython 3.7b1.

    This decorator works exactly the same as contextlib.contextmanager(), but for async with.

    Import it from mode.utils.contexts:

    from mode.utils.contexts import asynccontextmanager
    
    @asynccontextmanager
    async def connection_or_default(conn: Connection = None) -> Connection:
        if connection is None:
            async with connection_pool.acquire():
                yield
        else:
            yield connection
    
    async def main():
        async with connection_or_default() as connection:
            ...
    
  • Adds AsyncExitStack from CPython 3.7b1

    This works like contextlib.ExitStack, but for asynchronous context managers used with async with.

  • Logging: Worker debug log messages are now colored blue when colors are enabled.

1.5.0

release-date

2018-01-04 03:43 P.M PST

release-by

Ask Solem

  • Service: Adds new await self.add_context(context)

    This adds a new context manager to be entered when the service starts, and exited once the service exits.

    The context manager can be either a typing.AsyncContextManager (async with) or a regular typing.ContextManager (with).

  • Service: Added await self.add_runtime_dependency() which unlike add_dependency starts the dependent service if the self is already started.

  • Worker: Now supports a new console_port argument to specify a port for the aiomonitor console, different than the default (50101).

    Note

    The aiomonitor console is only started when Worker(debug=True, ...) is set.

1.4.0

release-date

2017-12-21 09:50 A.M PST

release-by

Ask Solem

  • Worker: Add support for parameterized logging handlers.

    Contributed by Prithvi Narasimhan.

1.3.0

release-date

2017-12-04 01:17 P.M PST

release-by

Ask Solem

  • Now supports color output in logs when logging to a terminal.

  • Now depends on colorlog

  • Added mode.Signal: async. implementation of the observer pattern (think Django signals).

  • DependencyGraph is now a generic type: DependencyGraph[int]

  • Node is now a generic type: Node[Service].

1.2.1

release-date

2017-11-06 04:50 P.M PST

release-by

Ask Solem

  • Service: Subclasses can now override a Service.task method.

    Previously it would unexpectedly start two tasks: the task defined in the superclass and the task defined in the subclass.

1.2.0

release-date

2017-11-02 03:17 P.M PDT

release-by

Ask Solem

  • Renames PoisonpillSupervisor to CrashingSupervisor.

  • Child services now stopped even if not fully started.

    Previously child_service.stop() would not be called if child_service.start() never completed, but as a service might be in the process of starting other child services, we need to call stop even if not fully started.

1.1.1

release-date

2017-10-25 04:34 P.M PDT

release-by

Ask Solem

  • Added alternative event loop implementations: eventlet, gevent, uvloop.

    E.g. to use gevent as the event loop, install mode using:

    $ pip install mode[gevent]
    

    and add this line to the top of your worker entrypoint module:

    import mode.loop
    mode.loop.use('gevent')
    
  • Service: More fixes for the weird __init_subclass__ behavior only seen in Python 3.6.3.

  • ServiceThread: Now propagates errors raised in the thread to the main thread.

1.1.0

release-date

2017-10-19 01:35 P.M PDT

release-by

Ask Solem

  • ServiceThread: Now inherits from Service, and uses loop.run_in_executor() to start the service as a thread.

  • setup_logging: filename argument is now respected.

1.0.2

release-date

2017-10-10 01:51 P.M PDT

release-by

Ask Solem

  • Adds support for Python 3.6.0

  • Adds backports of typing improvements in CPython 3.6.1 to mode.utils.compat: AsyncContextManager, ChainMap, Counter, and Deque.

  • Supervisor.add and .discard now takes an arbitrary number of services to add/discard as star arguments.

  • Fixed typo in example: Service.task -> mode.Service.task.

    Contributed by Xu Jing.

1.0.1

release-date

2017-10-05 02:53 P.M PDT

release-by

Ask Solem

  • Fixes compatibility with Python 3.6.3.

    Python 3.6.3 badly broke __init_subclass__, in such a way that any class attribute set is set for all subclasses.

1.0.0

release-date

2017-10-04 01:29 P.M PDT

release-by

Ask Solem

  • Initial release