fix: bound Timestamp date comparisons to the whole UTC day

[vishal-bala]

What this fixes

Timestamp treats a bare date as a whole calendar day in ==/!=, but not in the four comparison operators. __gt__, __lt__, __ge__ and __le__ all call _convert_to_timestamp(other) without end_date=True, and for a bare date that function combines with time.min — the start of the day — regardless of which operator is asking. Two of the four need the end of the day instead:

from datetime import date
from redisvl.query.filter import Timestamp
for op in ['>', '<', '>=', '<=']:
    print(op, eval(f'Timestamp("x") {op} date(2023,3,17)'))

Filter	Before	After
> date(2023,3,17)	@x:[(1679011200.0 +inf]	@x:[(1679097599.999999 +inf]	❌ → ✅
< date(2023,3,17)	@x:[-inf (1679011200.0]	unchanged	✅
>= date(2023,3,17)	@x:[1679011200.0 +inf]	unchanged	✅
<= date(2023,3,17)	@x:[-inf 1679011200.0]	@x:[-inf 1679097599.999999]	❌ → ✅

So > date(d) behaved like >= date(d), matching nearly all of day d that the caller asked to exclude, and <= date(d) behaved like < date(d), matching almost none of the day the caller asked to include. Both fail silently — a filter that returns the wrong rows, with no error or warning.

>= and < were correct only coincidentally: both legitimately anchor to the start of the day, so the missing argument never mattered for them.

Date-only ISO strings ("2023-03-17") are affected identically, since _is_date puts them on the same footing.

The fix

_convert_to_timestamp(value, end_date=True) already produces the 23:59:59.999999 bound — it's what between() uses for its upper end — so __gt__ and __le__ now pass it, reusing the existing mechanism rather than adding a second way to compute a day boundary. Resulting semantics for a bare date d:

• > d — after the end of UTC day d; the day is excluded
• <= d — through the end of UTC day d; the day is included
• >= d / < d — the start of UTC day d, unchanged

One extra step was needed for date-only strings. They arrive as str, where fromisoformat turns "2023-03-17" into a midnight datetime — which then fails the isinstance(value, date) and not isinstance(value, datetime) check and skips the branch that reads end_date altogether. So end_date=True alone would have fixed date objects and silently done nothing for strings. The new _as_date helper coerces a date-only string to a date, and it runs inside _convert_to_timestamp rather than at the call sites, which matters — see below.

The datetime, full-ISO-string and Unix-timestamp paths are untouched: end_date is only consulted for bare dates, and ints/floats return early. "2023-03-17T00:00:00" still resolves to a midnight instant, not a day range — a date-only string and a midnight datetime string remain distinct.

between() is fixed too

An earlier revision of this PR applied _as_date at the two operator call sites. That left between() — the third caller passing end_date=True — still short-circuiting:

Timestamp("f") <= "2023-03-19"                       # [-inf 1679270399.999999]  end of day
Timestamp("f").between("2023-03-17", "2023-03-19")   # [... 1679184000.0]        MIDNIGHT
Timestamp("f").between(date(2023,3,17), date(2023,3,19))  # [... 1679270399.999999]  end of day

Before this PR those agreed (all anchored to midnight — consistently wrong). Fixing only the operators would have made between() disagree with <= on identical input, and disagree with itself depending on whether you passed date(2023, 3, 19) or "2023-03-19". Doing the coercion once inside _convert_to_timestamp fixes all three call sites and means future end_date=True callers can't forget it.

Verified across 52 input/operator combinations: identical output to the call-site version everywhere except the two between()-with-date-string cases, which now correctly reach end-of-day.

Consistent error for a date-shaped non-date

_is_date_only matches on the digit pattern (^\d{4}-\d{2}-\d{2}$) alone, so "2023-02-30" reached strptime and raised its raw message from >/<= while </>= raised the curated one. _as_date now falls through on ValueError, so all four operators report String timestamp must be in ISO format: 2023-02-30 alike. Still a ValueError in every case, so no caller's except clause changes.

Docs

docs/api/filter.rst autoclassed every sibling filter — Tag, Text, Num, Geo, GeoRadius, FilterExpression — but not Timestamp, so none of these docstrings reached docs.redisvl.com. Added the missing block. Confirmed with a sphinx build that the class, all six operators and between() now render, and that the private helpers stay out.

Also documented between()'s whole-day semantics and its previously undocumented inclusive argument, and _convert_to_timestamp's end_date argument.

Tests

test_timestamp_operators exercised all four operators with a full datetime only, never a bare date — which is why this went unnoticed. Added:

• test_timestamp_comparison_operators_with_date — all four operators × {date object, date-only string}, with bounds hard-coded against 2023-03-17T00:00:00Z / T23:59:59.999999Z so the test cannot drift along with the implementation.
• test_timestamp_comparison_date_bounds_are_utc_days — the same matrix across five dates (ordinary, DST spring-forward, DST fall-back, the Unix epoch, past the signed 32-bit rollover) and five zones, using calendar.timegm as a timezone-independent oracle sharing no code with the path under test. Zones include the non-hour offsets Asia/Kathmandu (+05:45) and Pacific/Chatham (+12:45/+13:45), which an hour-granularity mistake would slip past. CI sets no TZ and GitHub runners default to UTC, so this is what would catch a regression to local-day bounds.
• test_timestamp_between_date_only_string_matches_date_object — pins the between() fix above.
• test_timestamp_date_shaped_but_invalid_string — 12 cases, four operators × three malformed-but-date-shaped inputs.

The six original cases fail without the fix (verified by stashing the filter.py change). tests/unit/test_filter.py: 87 pass, under both TZ unset and TZ=Asia/Kathmandu. Full unit suite: 1011 pass (the 2 failures are missing optional sentence-transformers in my venv, unrelated). make format, make check-types and pre-commit clean.

Release note

Fixed: Timestamp comparison filters now treat a bare date (or a date-only ISO string such as "2023-03-17") as the whole UTC calendar day. Previously > date(d) behaved like >= and matched nearly all of day d, and <= date(d) behaved like < and matched almost none of it. between() with a date-only string upper bound likewise stopped at midnight instead of covering the closing day. >= date(d) and < date(d) are unchanged, as are all filters built from datetime objects, full ISO datetime strings, or Unix timestamps.

This is a user-visible behavior change. Of the four operators, <= is the only one that now matches a larger row set (up to one extra calendar day); > matches a smaller one; < and >= are byte-identical. between() with a date-only string end bound also widens by up to a day. So a caller who used <= date(d) as a hard cutoff — a subscription end date, a retention or legal-hold boundary — will now see records from that final UTC day that were previously excluded. In every case the new result is what the documented semantics call for, but the widening is worth checking for at upgrade time.

Code that wants the old start-of-day bound explicitly can pass a datetime:

Timestamp("x") > datetime.combine(d, time.min, tzinfo=timezone.utc)

Relationship to #666

#666 has now merged (3707059) and this branch is rebased on top of it. It fixed a different defect in the same class: the ==/!= date branches anchored to the host's local day rather than the UTC day. The two changes are independent — the comparison operators go through _convert_to_timestamp, which already anchored bare dates to UTC, so this branch needed no UTC fix and makes none.

Together the two land the full contract. All six date forms now agree and are identical in every zone — verified under UTC, Europe/Berlin and Asia/Kathmandu:

Filter	Bound
< date(2023,3,17)	@x:[-inf (1679011200.0]
== date(2023,3,17)	@x:[1679011200.0 1679097599.999999]
> date(2023,3,17)	@x:[(1679097599.999999 +inf]
>= date(2023,3,17)	@x:[1679011200.0 +inf]
<= date(2023,3,17)	@x:[-inf 1679097599.999999]
between("2023-03-17", "2023-03-17")	@x:[1679011200.0 1679097599.999999]

<, == and > now tile the timeline exactly, with no gap and no overlap — which was not true of either PR alone. The rebase conflict was a single hunk (both PRs add imports to tests/unit/test_filter.py); #666's test_timestamp_date_bounds_are_utc_regardless_of_local_timezone and this PR's tests coexist, 87 passing in that file.

One consistency gap is deliberately left alone: __eq__/__ne__ still do their own inline strptime for date-only strings rather than going through _as_date, so they raise the raw strptime message for a date-shaped non-date like "2023-02-30" where the four comparison operators now raise the curated String timestamp must be in ISO format. Routing those two through _as_date would remove the duplication and unify the error, but it touches lines #666 just rewrote, so it belongs in a follow-up rather than here.