Skip to content

[Tracking] Supabase prod security and performance review, 2026-09-28 #327

Open
12 of 16 issues completed
Open

Description

@HMarzban

Key: SPR-00 · Type: tracking · Blocked by: none · Checked against: 0b302b2

Purpose

This issue tracks the review of the Supabase prod project on 2026-09-28. The review covered security and performance. It lists the public issues that the review filed, and holds one flag.

On 2026-09-28 at 09:29Z, the Supabase security linter flagged 7 ERROR items on prod (Postgres 15.8.1.073). The root cause is in the repo. The sweeps in packages/supabase/scripts/29-lint-hardening.sql §3, §4, §5 and §8 have no paired migration, so prod never ran them.

Two defaults keep each new prod object open:

  • Prod pg_default_acl (role postgres, schema public) gives anon and authenticated ALL on new tables and EXECUTE on new functions.
  • CREATE FUNCTION also grants EXECUTE to PUBLIC. A REVOKE ... FROM anon does not remove that grant, and anon inherits it.

Each child issue copies the facts it needs, so an agent can work from the repo and the issue alone.

What was applied to prod on 2026-09-28

At about 13:38Z, an agent applied an emergency fix to prod through the SQL connector. The maintainer approved it. It changed grants and RLS only. It changed no function body.

  • Transaction 1 (tables):
    • RLS was off on document_views_2026_03, _04, _05 and PushSentLog. Now RLS is on for every document_views partition, the parent, document_view_stats, document_views_daily, email_bounces and PushSentLog.
    • anon and authenticated lost every grant on those tables. service_role keeps its access.
    • anon lost every grant on admin_users, so the is_admin revoke below breaks no guest read.
    • anon lost table-level SELECT on public.users. It got a column grant back without email, so guests can still read username.
  • Transaction 2 (functions):
    • 27 functions became service_role only. Examples: purge_document_footprint, consume_push_queue, get_email_bounces, and the four prod-only functions.
    • is_admin was revoked from anon. authenticated keeps it.

Results after the change:

  • Security linter: 0 ERROR (was 7). Lint 0028 fell from 72 to 47 rows.
  • The built-in asserts in both transactions passed. Cron jobs 2, 8, 17, 25 and 28 ran and succeeded afterwards.
  • get_channel_aggregate_data was not touched, so guest chat still works.

Prod now differs from the repo. A fresh db reset or a self-hosted install still starts open. Any later migration that drops and re-creates one of these functions resets its grants to the open defaults. #314 ports the slice into the repo.

Other facts at the time of the review:

  • Migration history: prod history ends at 20260921120000. The repo has three later migrations: 20260922120000 (commit f6ed0a22c), 20260928120000 (commit e4113fd9d) and 20260928130000 (commit 3ef541d8a). Ship the three migrations that prod's history lacks #313 ships them.
  • Performance: all SQL together uses 35.0 s of DB time per hour, 0.97% of one core, in a 69.8-minute prod window. Realtime list_changes takes 78% of that. The database is 62 MB. Supabase bills compute by instance size, so query work saves $0.
  • Next partition: the monthly cron (1st of the month, 00:05 UTC) runs create_document_views_partitions. At review time, its 1 October 2026 run was due to create document_views_2027_01 with the open default grants.
    • On prod, the ensure_rls event trigger turns RLS on, so the new partition stays closed. The repo has no such trigger.
    • In the repo, create_document_views_partitions in packages/supabase/scripts/09-document-views.sql (~105) turns RLS on but revokes no grant. The first DO block (~66) does neither. Make new document_views partitions start closed #315 owns both gaps.

The flag

Flag: amber. Both issues in the red list, #313 and #314, are closed. #322, #323, #324, #325 and #755 are still open.

The red list is #313 and #314. #314 covers a prod state the repo cannot rebuild, and #313 must land before it.

Update the flag when an issue closes. Move to amber only when every issue in the red list is closed.

Benchmark ledger

Condition: Supabase production project, Postgres 15.8.1.073 until #322 lands. The maintainer reads each row in the SQL editor or Advisors > Security, with the date. An agent in the repo updates one row when a child posts a number. Quote numbers only from their #763 row, which links each child's Measured comment.

Child Metric Before After Status
#313-#321 Security linter ERROR rows (target 0) 7 (2026-09-28) 0 (2026-11-03) done
#318 Lint 0028 rows (target 0, or each row ruled) 72 (2026-09-28) 47 after the emergency fix; not re-read after #318 re-read owed: maintainer, same session as the #755 after-read
#316, #317 check-grants.sql rows (target 0) 4 (2026-10-19) 0 (2026-11-03) done
#755 Notification RPCs anon can EXECUTE (target 0 of 3; authenticated stays 3 of 3; service_role recorded, no target) 3 of 3 (2026-11-10) not measured yet not built
#322 vulnerable_postgres_version rows (target 0) 1 (2026-09-28) not measured yet waits on #325, #324 and #755
#322 Upgrade outage seconds (target 3,600 or less) n/a not measured yet not run
#323 get_cron_job_health calls per hour (target 30-50) 120 40 (2026-11-07, Measured) pass; guard passed 2026-11-11; state-history line by 2026-11-17
#323 get_cron_job_health DB ms per hour (target about 1,245) 3,736 1,294.8 (2026-11-07) pass
#325 list_changes spike days in 7 (target 0 of 7) unknown; worst 14,561.2 ms (read 2026-10-25) not measured yet window not started
#324 Host memory, swap, CPU over 7 days (median and range) not measured yet not measured yet window not started

Claim (true today): "A production review of the docs.plus Supabase database closed 7 of 7 security linter errors."

Add this sentence only after the #755 after-read:

"Signed-out visitors can run 0 of 3 notification functions (was 3)."

The grant guard cannot see invoker functions, so it gives no claim about them. Mirror the claim on #763.

The issues

Urgent decisions

Security repo fix

Security rulings

Platform

Performance

Records

Tracked privately

Five further security items (SPR-01, SPR-08, SPR-11, SPR-12, SPR-13) follow SECURITY.md. Each is filed here after it is fixed on prod.

Order and dependencies

  1. Push the invoker-RPC migration (Revoke anon and public EXECUTE on three invoker notification RPCs #755).
  2. Wait for its clean 24 h GlitchTip read. Then, on one day, run Find the cause of the 13-15 s realtime list_changes spikes #325 step 4 and start Measure prod memory, swap and CPU for 7 days before any compute change #324.
  3. On day 8 or later, run Upgrade Supabase Postgres from 15.8.1.073 to 17 #322 steps 4-7.
  4. Upgrade Supabase Postgres from 15.8.1.073 to 17 #322 step 8.

#323: the guard passed on 2026-11-11. The maintainer posts the Grafana state-history line for cron-stale-frequent (2026-11-03 to 2026-11-10) by 2026-11-17, then closes it.

#322 and the #325 reset both reset pg_stat_statements. So the #325 reads from 2026-10-25 are posted first, and #322 step 1b saves its query-cost before.

Not yet owned

Green needs a ruling or an owner for each item the review found. Each item now has one:

Moved out: #570 (v2.4.0) owns the pg_default_acl rows for role postgres in schema storage.

What we decided not to do

The review measured and rejected 14 query ideas and parked 4 more. Examples: moving chat postgres_changes to broadcast, the advisor's channel_members (channel_id) index, and a Micro downgrade before memory data exists. The numbers and the reopen triggers are in #326. Do not re-propose one of these ideas without new numbers.

Activity

  1. 11 remaining items

  2. HMarzban commented on Oct 24, 2026

    @HMarzban
    CollaboratorAuthor

    Maintainer ruling, 2026-10-24

    1. The three SECURITY INVOKER RPCs with browser callers and no explicit grant (get_unread_notifications_paginated, get_channel_notif_state, get_workspace_notifications): not decided yet. This needs a deeper look first. It moves to the end of milestone v2.3.0, as one of the last tasks.
    2. Drop get_bookmark_count and admin_get_user_notification_subs: yes. Neither has a caller. Drop the unused get_bookmark_count and admin_get_user_notification_subs functions #605 owns the drop.
  3. HMarzban commented on Nov 3, 2026

    @HMarzban
    CollaboratorAuthor

    Status, 2026-11-03. The child #605 is pushed in eb4e7fb53. Its migration 20261025120000 still needs a manual prod push. These items are left:

    #326 stays open as a record. The flag stays amber.

  4. HMarzban commented on Nov 3, 2026

    @HMarzban
    CollaboratorAuthor

    Status, 2026-11-03 (after the 1b05cb2 deploy). #605 is closed, and its migration 20261025120000 is on production. Every pending Supabase migration was pushed on 2026-11-03. check-grants.sql then returned 0 rows (close comment on #605). That meets the "#317 guard runs clean" part of Green.

    The flag stays amber, because #322, #323, #324 and #325 are still open. Left:

    #326 stays open as a record.

  5. modified the milestones: v2.3.0, v2.4.0 on Nov 4, 2026
  6. HMarzban commented on Nov 4, 2026

    @HMarzban
    CollaboratorAuthor

    Maintainer ruling, 2026-11-04

    Moved from milestone v2.3.0 to milestone v2.4.0, from the v2.3.0 open-issue review. #313–#321 are closed. The open children are #322–#326, which moved off v2.3.0, so the tracker moves with them.

    The issue stays open. Its remaining items are unchanged.

  7. modified the milestones: v2.4.0, v2.3.6 on Nov 9, 2026
  8. HMarzban commented on Nov 10, 2026

    @HMarzban
    CollaboratorAuthor

    Maintainer ruling, 2026-11-10

    On the three SECURITY INVOKER RPCs with browser callers: get_unread_notifications_paginated, get_channel_notif_state and get_workspace_notifications. Answer: "agree".

    The deeper look, read from code and not checked on production:

    • All three are SECURITY INVOKER, because none sets a security clause (packages/supabase/scripts/10-functions.sql:552 and :711, packages/supabase/scripts/07-1-notification-functions.sql:20).
    • The local seed grants all three to authenticated through the user_facing_names list in packages/supabase/scripts/29-lint-hardening.sql §6.
    • packages/supabase/scripts/check-grants.sql does not cover invoker functions. So its clean production run on 2026-11-03 says nothing about these three.

    The ruling:

    1. First, run one read-only has_function_privilege query on production, for anon and authenticated, on each of the three.
    2. If anon can run any of them, add one paired change in scripts/ and a migration: grant EXECUTE to authenticated, and revoke it from anon and public.
    3. If anon cannot, close this item as covered.
  9. HMarzban commented on Nov 10, 2026

    @HMarzban
    CollaboratorAuthor

    Result, 2026-11-10. The maintainer ran the read-only check in the production SQL editor.

    Function anon authenticated
    get_channel_notif_state true true
    get_unread_notifications_paginated true true
    get_workspace_notifications true true

    anon can run all three. So step 2 of the 2026-11-10 ruling applies: one paired change in scripts/ and a migration that grants EXECUTE to authenticated and revokes it from anon and public.

    Callers, read from code on 2026-11-10:

    • get_channel_notif_state: apps/webapp/src/components/chatroom/hooks/useNotificationToggle.ts. The query runs only with a signed-in userId (enabled: Boolean(channelId && userId)).
    • get_unread_notifications_paginated: apps/webapp/src/components/notificationPanel/feed/useNotificationPanelFeed.ts. Every call returns early without a userId.
    • get_workspace_notifications: only apps/webapp/src/api/notification/getLastReadNotification.ts, and nothing imports that wrapper except the api/notification barrel. It has no live caller.
    • No other app, extension or package calls them. apps/webapp/cypress/support/chatroomFixtures.ts stubs them.

    So no signed-out path breaks when anon loses EXECUTE.

    Notes for the builder:

    • The functions are SECURITY INVOKER, so the revoke must name public too. has_function_privilege('anon', …) is also true through the PUBLIC grant.
    • The grant to authenticated already sits in 29-lint-hardening.sql §6. The migration must carry it too, because production never ran the local sweep.
    • Run packages/supabase/scripts/check-grants.sql and this same query on production after the push. Expect anon false and authenticated true on all three.
    • get_workspace_notifications has no live caller. Dropping it with its wrapper is a separate question. It is not part of this change.
  10. HMarzban commented on Nov 11, 2026

    @HMarzban
    CollaboratorAuthor

    Maintainer ruling, 2026-11-11

    This tracker stays in milestone v2.3.6, the benchmark milestone. This voids the move to v2.4.0 in the ruling of 2026-11-04.

    In scope: #322, #323, #324, #325 and the new invoker-RPC child #755. Out of scope: #326 (record) and #570 (v2.4.0).

    The 2026-11-10 ruling on the three SECURITY INVOKER notification RPCs now has an owner, #755. anon could run 3 of 3 on 2026-11-10. The child is pushed before the #325 reset, so no db push falls inside the #325 window.

    The body now has a Benchmark ledger. An agent updates one row when a child posts a number.

  11. modified the milestones: v2.3.6, v2.3.7 on Nov 12, 2026
  12. HMarzban commented on Nov 12, 2026

    @HMarzban
    CollaboratorAuthor

    Moved to v2.3.7, 2026-11-12 (maintainer ruling). v2.3.6 ships on 2026-11-14 with only finished, measured work, and its children #322, #324 and #325 need 7-day windows. The before values and the plan in this issue stay as they are.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    DevOpsSecuritySecurity, access control, and data exposure

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions