botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.
10:41:25 [ERROR] arroyo.processing.processor: Caught exception, shutting down...
10:41:38 [INFO] arroyo.processing.processor: Closing <arroyo.backends.kafka.consumer.KafkaConsumer object at 0x7f7b8e59f0b0>...
10:41:38 [INFO] arroyo.processing.processor: Partitions to revoke: [Partition(topic=Topic(name='ingest-replay-recordings'), index=0)]
10:41:38 [INFO] arroyo.processing.processor: Partition revocation complete.
10:41:38 [INFO] arroyo.processing.processor: Processor terminated
Traceback (most recent call last):
  File "/.venv/bin/sentry", line 4, in <module>
    raise SystemExit(main())
                     ^^^^^^
  File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main
    func(**kwargs)
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/decorators.py", line 83, in inner
    return ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
    return ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/commands/run.py", line 455, in basic_consumer
    run_processor_with_signals(processor, consumer_name)
  File "/usr/src/sentry/src/sentry/utils/kafka.py", line 46, in run_processor_with_signals
    processor.run()
  File "/.venv/lib/python3.12/site-packages/arroyo/processing/processor.py", line 323, in run
    self._run_once()
  File "/.venv/lib/python3.12/site-packages/arroyo/processing/processor.py", line 397, in _run_once
    self.__processing_strategy.poll()
  File "/.venv/lib/python3.12/site-packages/arroyo/processing/strategies/guard.py", line 101, in poll
    self.__inner_strategy.poll()
  File "/.venv/lib/python3.12/site-packages/arroyo/processing/strategies/run_task.py", line 55, in poll
    self.__next_step.poll()
  File "/.venv/lib/python3.12/site-packages/arroyo/processing/strategies/guard.py", line 37, in poll
    self.__next_step.poll()
  File "/.venv/lib/python3.12/site-packages/arroyo/processing/strategies/run_task_in_threads.py", line 107, in poll
    result = future.result()
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/replays/consumers/recording.py", line 129, in process_message_threaded
    ingest_recording(message_dict, context.transaction, context.isolation_scope)
  File "/usr/src/sentry/src/sentry/utils/metrics.py", line 229, in inner
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/replays/usecases/ingest/__init__.py", line 90, in ingest_recording
    _ingest_recording(message, transaction)
  File "/usr/src/sentry/src/sentry/replays/usecases/ingest/__init__.py", line 154, in _ingest_recording
    storage_kv.set(make_recording_filename(segment_data), compressed_segment)
  File "/usr/src/sentry/src/sentry/utils/metrics.py", line 229, in inner
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/replays/lib/storage.py", line 209, in set
    storage.save(key, BytesIO(value))
  File "/.venv/lib/python3.12/site-packages/django/core/files/storage/base.py", line 49, in save
    name = self._save(name, content)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/filestore/s3.py", line 498, in _save
    self._save_content(obj, content, parameters=parameters)
  File "/usr/src/sentry/src/sentry/filestore/s3.py", line 513, in _save_content
    obj.upload_fileobj(content, ExtraArgs=put_parameters)
  File "/.venv/lib/python3.12/site-packages/boto3/s3/inject.py", line 731, in object_upload_fileobj
    return self.meta.client.upload_fileobj(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/boto3/s3/inject.py", line 642, in upload_fileobj
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/s3transfer/futures.py", line 266, in result
    raise self._exception
  File "/.venv/lib/python3.12/site-packages/s3transfer/tasks.py", line 139, in __call__
    return self._execute_main(kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/s3transfer/tasks.py", line 162, in _execute_main
    return_value = self._main(**kwargs)
                   ^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/s3transfer/upload.py", line 764, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/.venv/lib/python3.12/site-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/botocore/client.py", line 1021, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
root@sentry:/home/ubuntu/sentry/sentry#