mbox series

[RFC,0/3] Call for help for prserv selftests

Message ID 20240416171945.3799445-1-michael.opdenacker@bootlin.com
Headers show
Series Call for help for prserv selftests | expand

Message

Michael Opdenacker April 16, 2024, 5:19 p.m. UTC
From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Hi Joshua and BitBake contributors,

I'm trying to implement the first BitBake selftests for the PR server.
I tried to replicate what is done by the hashserv selftests, but I
still can't seem to connect to the server I started.

I'm attaching my code (especially the 3rd patch) as a patch series.

I won't give up, but if you have clues about what could be wrong, they
would be much appreciated!

Here's the output I'm getting:

mike@xps:~/work/yocto/poky/build$ bitbake-selftest prserv.tests.PRCommonTests
E
Stdout:
SERVER STARTED WITH ADDRESS: 127.0.0.1:33399
CLIENT CONNECTED TO: 127.0.0.1:33399
Error talking to server: [Errno 104] Connection reset by peer
Error talking to server: [Errno 111] Connect call failed ('127.0.0.1', 33399)
Error talking to server: [Errno 111] Connect call failed ('127.0.0.1', 33399)
Error talking to server: [Errno 111] Connect call failed ('127.0.0.1', 33399)

======================================================================
ERROR: test_create_pr (prserv.tests.PRCommonTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mike/work/yocto/poky/bitbake/lib/prserv/tests.py", line 81, in test_create_pr
    result = self.client.test_pr("dummy-1.0-r0", "core2-64", "51bf8189dbe9ea81fa6dd89608bf19380c437a9cf12f6c6239887801ba4ab4a5")
  File "/home/mike/work/yocto/poky/bitbake/lib/bb/asyncrpc/client.py", line 201, in wrapper
    return self.loop.run_until_complete(downcall(*args, **kwargs))
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/mike/work/yocto/poky/bitbake/lib/prserv/client.py", line 25, in test_pr
    response = await self.invoke(
  File "/home/mike/work/yocto/poky/bitbake/lib/bb/asyncrpc/client.py", line 164, in invoke
    result = await self._send_wrapper(proc)
  File "/home/mike/work/yocto/poky/bitbake/lib/bb/asyncrpc/client.py", line 151, in _send_wrapper
    raise e
  File "/home/mike/work/yocto/poky/bitbake/lib/bb/asyncrpc/client.py", line 138, in _send_wrapper
    await self.connect()
  File "/home/mike/work/yocto/poky/bitbake/lib/bb/asyncrpc/client.py", line 123, in connect
    self.socket = await self._connect_sock()
  File "/home/mike/work/yocto/poky/bitbake/lib/bb/asyncrpc/client.py", line 64, in connect_sock
    reader, writer = await asyncio.open_connection(address, port)
  File "/usr/lib/python3.10/asyncio/streams.py", line 48, in open_connection
    transport, _ = await loop.create_connection(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1076, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1060, in create_connection
    sock = await self._connect_sock(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 969, in _connect_sock
    await self.sock_connect(sock, address)
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 501, in sock_connect
    return await fut
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 541, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 33399)

Stdout:
SERVER STARTED WITH ADDRESS: 127.0.0.1:33399
CLIENT CONNECTED TO: 127.0.0.1:33399
Error talking to server: [Errno 104] Connection reset by peer
Error talking to server: [Errno 111] Connect call failed ('127.0.0.1', 33399)
Error talking to server: [Errno 111] Connect call failed ('127.0.0.1', 33399)
Error talking to server: [Errno 111] Connect call failed ('127.0.0.1', 33399)

----------------------------------------------------------------------
Ran 1 test in 0.015s

FAILED (errors=1)

---

Thanks in advance
Michael.

Michael Opdenacker (3):
  prserv: add "upstream" server support
  prserv: add "history" argument to "get-pr" request
  prserv: start bitbake selftests

 bin/bitbake-prserv     |  15 ++++-
 bin/bitbake-selftest   |   2 +
 lib/prserv/__init__.py |  39 +++++++++++++
 lib/prserv/client.py   |   5 +-
 lib/prserv/db.py       | 127 ++++++++++++++++++++---------------------
 lib/prserv/serv.py     |  98 ++++++++++++++++++++++++++-----
 lib/prserv/tests.py    |  84 +++++++++++++++++++++++++++
 7 files changed, 289 insertions(+), 81 deletions(-)
 create mode 100644 bitbake/lib/prserv/tests.py

Comments

Michael Opdenacker April 17, 2024, 2:34 p.m. UTC | #1
Greetings,

I eventually found out why I didn't have any logging...

On 4/16/24 at 19:19, Michael Opdenacker via lists.openembedded.org wrote:
> +class PRTestSetup(object):
> +
> +    server_index = 0
> +    client_index = 0
> +
> +    def start_server(self, dbpath=None, upstream=None, read_only=False, prefunc=server_prefunc):
> +
> +        self.server_index += 1
> +
> +        if dbpath is None:
> +            dbpath = self.make_dbpath()
> +
> +        def cleanup_server(server):
> +            if server.process.exitcode is not None:
> +                return
> +
> +            server.process.terminate()
> +            server.process.join()
> +
> +        server = create_server(socket.gethostbyname("localhost") + ":0",
> +                               dbpath,
> +                               upstream=upstream,
> +                               read_only=read_only)
> +
> +        server.serve_as_process(prefunc=prefunc, args=(self.server_index))

I was missing a comma at the end of "args" in the above line. It should 
have been:
server.serve_as_process(prefunc=prefunc, args=(self.server_index,)

Without the comma, the prefunc function was never called and I got no 
logging.

Now, I'm studying the server logs :)
Thanks Joshua for the tips!
Cheers
Michael.
Michael Opdenacker April 17, 2024, 2:42 p.m. UTC | #2
On 4/17/24 at 16:34, Michael Opdenacker via lists.openembedded.org wrote:
>> + server.serve_as_process(prefunc=prefunc, args=(self.server_index))
>
> I was missing a comma at the end of "args" in the above line. It 
> should have been:
> server.serve_as_process(prefunc=prefunc, args=(self.server_index,)
>
> Without the comma, the prefunc function was never called and I got no 
> logging.


And worse, the asyncrpc server process didn't get started, though the 
rest of the server was (like creating the database).
My first test is now successful and I can implement many more :)

Cheers
Michael.