mbox series

[bitbake-devel,RFC,v2,00/18] Bitbake Hash Server WebSockets, Alternate Database Backend, and User Management

Message ID 20231012221655.632637-1-JPEWhacker@gmail.com
Headers show
Series Bitbake Hash Server WebSockets, Alternate Database Backend, and User Management | expand

Message

Joshua Watt Oct. 12, 2023, 10:16 p.m. UTC 
This patch series reworks the bitbake asyncrpc API to add a WebSockets
implementation for both the client and server. The hash equivalence
server is updated to allow using this new API (the PR server can also be
updated in the future if desired).

In addition, the database backed for the hash equivalence server is
abstracted so that sqlalchemy can optionally be used instead of sqlite.
This allows using "big metal" databases as the backend, which allows the
hash equivalence server to scale to a large number of queries.

Note that both websockets and sqlalchemy require 3rd party python
modules to function. However, these modules are optional unless the user
desires to use the APIs.

Also, user management is added. This allows user accounts to be
registered with the server and users can be given permissions to do
certain operations on the server. Users are not (necessarily) required
to login to access the server, as permissions can granted to anonymous
users. The default permissions will give anonymous users the same
permissions that they would have before user accounts were added so as
to retain backward compatibility, but server admins will likely want to
change this.

Joshua Watt (18):
  asyncrpc: Abstract sockets
  hashserv: Add websocket connection implementation
  asyncrpc: Add context manager API
  hashserv: tests: Add external database tests
  asyncrpc: Prefix log messages with client info
  bitbake-hashserv: Allow arguments from environment
  hashserv: Abstract database
  hashserv: Add SQLalchemy backend
  contrib: Update hashserv Dockerfile
  contrib: hashserv: Add docker-compose
  hashserv: Implement read-only version of "report" RPC
  asyncrpc: Add InvokeError
  asyncrpc: client: Prevent double closing of loop
  asyncrpc: client: Add disconnect API
  hashserv: Add user permissions
  hashserv: Add become-user API
  hashserv: Add db-usage API
  hashserv: Add database column query API

 bin/bitbake-hashclient               | 139 +++-
 bin/bitbake-hashserv                 | 129 +++-
 contrib/hashserv/Dockerfile          |  33 +-
 contrib/hashserv/docker-compose      |  11 +
 contrib/hashserv/docker-compose.yaml |  36 +
 contrib/hashserv/requirements.txt    |   5 +
 lib/bb/asyncrpc/__init__.py          |  33 +-
 lib/bb/asyncrpc/client.py            | 120 ++--
 lib/bb/asyncrpc/connection.py        | 146 ++++
 lib/bb/asyncrpc/exceptions.py        |  21 +
 lib/bb/asyncrpc/serv.py              | 354 ++++++----
 lib/hashserv/__init__.py             | 190 +++---
 lib/hashserv/client.py               | 151 ++++-
 lib/hashserv/server.py               | 950 ++++++++++++++++-----------
 lib/hashserv/sqlalchemy.py           | 427 ++++++++++++
 lib/hashserv/sqlite.py               | 391 +++++++++++
 lib/hashserv/tests.py                | 418 +++++++++++-
 lib/prserv/client.py                 |   8 +-
 lib/prserv/serv.py                   |  37 +-
 19 files changed, 2792 insertions(+), 807 deletions(-)
 create mode 100755 contrib/hashserv/docker-compose
 create mode 100644 contrib/hashserv/docker-compose.yaml
 create mode 100644 contrib/hashserv/requirements.txt
 create mode 100644 lib/bb/asyncrpc/connection.py
 create mode 100644 lib/bb/asyncrpc/exceptions.py
 create mode 100644 lib/hashserv/sqlalchemy.py
 create mode 100644 lib/hashserv/sqlite.py