Sujay Dongre
Work summary
Google Summer of Code 2026Final work summary

Kafka
Security Project

Let Ceph’s S3 gateway send bucket notifications to a secured Kafka cluster, whatever login method that cluster requires.

Contributor
Sujay Dongre
@sujay-d07
Organisation
Ceph
RADOS Gateway (RGW) · Bucket Notifications
Mentor
Yuval Lifshitz
Ceph RGW
Upstream
ceph/ceph
github.com/ceph/ceph
1e9997d9be42026-05-19rgw/kafka: add support for GSSAPI mechanism for Kafka Bucket Notification Endpoint·803f698200e2026-06-01rgw/testing: add GSSAPI testing in bucket notification tests·93a357c93c02026-06-12qa/rgw: Adding GSSAPI testing task for Bucket Notification Testing·d1a1f0941ef2026-07-10rgw/kafka: add support for OAuthbearer Mechanism for Kafka Bucket Notification Endpoint·c1b6a67f9c52026-07-10rgw/testing: add OAuthbearer Testing in Kafka Bucket Notification Tests·91bf47149442026-07-10qa/rgw: add OAuthbearer testing support in Teuthology·4b89f66ed6d2026-07-27rgw/kafka: add support for Fileless CA certificate passing in Kafka connection configuration·ffd848a0de52026-08-03rgw/testing: add fileless CA passing testing in Kafka Bucket Notification Test Suite·1e9997d9be42026-05-19rgw/kafka: add support for GSSAPI mechanism for Kafka Bucket Notification Endpoint·803f698200e2026-06-01rgw/testing: add GSSAPI testing in bucket notification tests·93a357c93c02026-06-12qa/rgw: Adding GSSAPI testing task for Bucket Notification Testing·d1a1f0941ef2026-07-10rgw/kafka: add support for OAuthbearer Mechanism for Kafka Bucket Notification Endpoint·c1b6a67f9c52026-07-10rgw/testing: add OAuthbearer Testing in Kafka Bucket Notification Tests·91bf47149442026-07-10qa/rgw: add OAuthbearer testing support in Teuthology·4b89f66ed6d2026-07-27rgw/kafka: add support for Fileless CA certificate passing in Kafka connection configuration·ffd848a0de52026-08-03rgw/testing: add fileless CA passing testing in Kafka Bucket Notification Test Suite·
01
Why this project

The problem

Ceph’s S3 gateway could already send bucket notifications to Kafka. It just could not do it securely, which ruled it out for most real deployments.
Before
  • RGW can send a message to Kafka whenever an object is uploaded or deleted. That already worked — but only over a connection with no encryption and no login.
  • Most Kafka clusters in production require a login. RGW could not perform the kinds of login they ask for.
  • That left operators with two bad choices: open an insecure port on their Kafka cluster, or go without notifications.
After
  • RGW can now log in to Kafka the three ways it previously could not: with a client certificate (mTLS), with a Kerberos ticket (GSSAPI), or with a token from an identity provider (OAuthBearer).
  • Kerberos and OAuthBearer are the two I wrote. Both hand the login off to an outside service, so no password is stored in the notification settings.
  • All of it is tested automatically. For Kerberos and OAuthBearer that meant adding a real Kerberos server and a real identity provider to Ceph’s test system, because neither existed there before.
02
Four deliverables

GSoC work

Three pull requests I wrote and one I reviewed. Each entry says what it lets Ceph do, then what I actually did. About ~1 500 lines in total, across the Kafka driver, the test system and the test suite.

mTLS

client certificatesReviewedMerged 24 May 2026#68771

rgw/kafka: add mTLS support (extends #61572)

Some Kafka clusters do not use passwords at all. Instead, every client has to present its own certificate as proof of identity. RGW can now do that.

What I did
  • Reviewed the patch over three weeks with my mentor, asking for changes twice before approving it.
  • Tested it on my own machine against a Kafka broker set to require client certificates.
  • Suggested how the test should be written. That suggestion was used as-is.

mTLS was on my project list, but my mentor wanted to leave it open for a new contributor rather than have me do it. Someone else wrote the patch and I took the reviewer seat.

The test shape I proposed in review
@pytest.mark.kafka_security_test
def test_notification_kafka_security_ssl_mtls():
    kafka_security('SSL', use_mtls=True)

Merged into test_bn.py with a use_mtls option added to the shared kafka_security() helper — exactly what I suggested.

Inline comments
9
on the diff
Changes requested
2
before approval
Final approval
24 May 2026

GSSAPI

KerberosWroteMerged 20 Jul 2026#69200

rgw/kafka: Adding missing support for GSSAPI bucket notification endpoint and relevant tests for it

Kafka clusters joined to Active Directory require Kerberos. Before this, those setups could not receive bucket notifications at all, because RGW had no way to log in. Now it can, using a keytab and principal set on the topic.

What I did
  • Added Kerberos login to RGW’s Kafka driver. The service name, principal and keytab are read from the topic settings and passed to librdkafka.
  • Made RGW keep a separate Kafka connection per Kerberos identity, so two topics logging in as different users cannot end up sharing one connection.
  • Added a cluster setting, rgw_kafka_sasl_kerberos_service_name, as the default that a topic can override.
  • Wrote qa/tasks/kerberos.py, which sets up a working Kerberos server in Ceph’s test system so the tests run against the real thing instead of a stand-in.
  • Added Valgrind suppressions for the Kerberos libraries, so the memory-checking tests still pass.

OAuthBearer

single sign-onWroteIn review#70100

rgw/kafka: Add OAuthbearer support and testing for Kafka bucket notifications

Instead of storing a password, RGW can now ask the organisation’s identity provider — the same system behind single sign-on — for a short-lived token, and use that token to connect to Kafka. Fixes tracker issue #74725.

What I did
  • Added OAuthBearer login to the Kafka driver and the topic API. The token URL, client ID, secret and optional scope are set on the topic; librdkafka then fetches and renews tokens by itself.
  • Treated the client secret like a password: never written to logs, and rejected if it is sent over an unencrypted connection, unless rgw_allow_notification_secrets_in_cleartext is turned on.
  • Wrote qa/tasks/dex.py, which runs a real identity provider (Dex) in Ceph’s test system so the tests use genuinely signed tokens.
  • Added tests over both encrypted and unencrypted connections.

Fileless CA

no file neededWroteIn review#70771

rgw/kafka: Add fileless CA certificate support for Kafka bucket notifications

To check that a Kafka broker is genuine, RGW needs a CA certificate. It used to have to be saved as a file on every RGW machine. On Kubernetes and Rook that certificate lives in a Secret, so writing it to disk was awkward. You can now put the certificate straight into the topic settings.

What I did
  • Added a ca-cert topic setting that takes the certificate contents and passes them to librdkafka in memory, with nothing written to disk.
  • ca-cert takes priority over ca-location if both are set, and requires use-ssl=true.
  • Added tests for the new path alongside the existing file-based one.
03
What RGW supports now

Mechanism matrix

You pick a configuration by setting attributes on the topic. The attribute names below are copied exactly from doc/radosgw/notifications.rst. Rows marked Pre-GSoC are what RGW could already do before my project — they are here to show what changed.
MechanismEncryptionAuthenticationTopic attributesMy contributionStatus
Plaintext
NoneNone
push-endpoint
Pre-GSoC
Baseline
SASL_PLAINTEXT · PLAIN
NonePassword
user-namepassword
Pre-GSoC
Baseline
SASL · SCRAM-SHA-256 / 512
None / TLSHashed password
user-namepasswordmechanism
Pre-GSoC
Baseline
SSL
TLSNone
use-sslca-location
Pre-GSoC
Baseline
SASL_SSL
TLSPassword
use-sslca-locationuser-namepasswordmechanism
Pre-GSoC
Baseline
mTLS
Written by a new contributor. I reviewed it and gave the final approval.
TLS (both sides)Client certificate
use-sslca-locationssl-certificate-locationssl-key-location
Reviewed
Merged#68771
GSSAPI
The login is handled by a Kerberos server.
Optional TLSKerberos ticket
mechanism=GSSAPIsasl-kerberos-service-namesasl-kerberos-principalsasl-kerberos-keytab
Implemented
Merged#69200
OAuthBearer
RGW supplies the client details once; librdkafka fetches and renews the token from then on.
Optional TLSOIDC token
mechanism=OAUTHBEARERsasl-oauthbearer-token-endpoint-urlsasl-oauthbearer-client-idsasl-oauthbearer-client-secretsasl-oauthbearer-scope
Implemented
In review#70100
Fileless CA certificate
Not a login method. ca-cert holds the certificate itself, and takes priority over ca-location.
TLS
use-ssl=trueca-cert
Implemented
In review#70771
04
Not in the plan

Extra bits:

Neither of these was part of my proposal. Kafka’s documentation explains Kafka’s own settings, Ceph’s documentation lists the topic attributes, and nothing joined the two up. Anyone running a secured Kafka cluster had nothing to follow.
ExtraPART 1
Awaiting merge#1183

Kafka Security for Bucket Notifications in Ceph RGW — Part 1: SASL, TLS, and mTLS

11 Aug 2026·ceph.io

The methods where the login details sit on the topic itself: password, hashed password, TLS, and client certificates. Covers the broker settings, how to generate the certificates, and how to move an existing topic across one step at a time.

ExtraPART 2
Awaiting merge#1191

Kafka Security for Bucket Notifications in Ceph RGW — Part 2: GSSAPI and OAuthBearer

23 Aug 2026·ceph.io

The two methods where an outside service handles the login: Kerberos and OIDC. Includes how to set up a test Kerberos realm and a test identity provider, plus a table of common errors and what causes each one.

05
How it ran

Timeline

Every date comes from commit or pull-request metadata, not from memory.
  1. REVIEW2026-05-06 → 05-24

    Reviewed the mTLS patch

    #68771

    Took the reviewer seat on #68771 so a new contributor could deliver the feature. Asked for changes twice, then approved it on the day it merged.

  2. CODING2026-05-19

    First Kerberos login

    RGW logged in to a Kafka broker with a Kerberos ticket for the first time.

  3. CODING2026-06-12

    Kerberos server in CI

    qa/tasks/kerberos.py sets up a working Kerberos server inside Ceph’s test system, so the Kerberos tests are not faking anything.

  4. CODING2026-07-10

    OAuthBearer opened

    #70100

    #70100: token-based login from end to end, with a real identity provider running in CI to issue the tokens.

  5. CODING2026-07-20

    GSSAPI merged

    Milestone#69200

    Kerberos became a supported way to send bucket notifications in Ceph. The largest piece of the project.

  6. CODING2026-08-03

    Fileless CA certificates

    #70771

    #70771: pass the CA certificate directly, instead of saving it as a file on every RGW machine.

  7. EXTRA2026-08-11

    Part 1 published

    #1183

    First of two guides on ceph.io. Not part of the proposal — I wrote it because none of this was documented anywhere.

  8. EXTRA2026-08-23

    Part 2 published

    Milestone#1191

    The Kerberos and OIDC guide, including a test setup to try it on and a table of common errors.

06
Honest closing position

Where it stands

Two merged, two still in review, and both blog posts submitted. The open ones have active review threads.