[tz] Extra transition for Europe/London with 2023d
Brooks Harris
brooks at edlmax.com
Sun Jan 7 20:01:32 UTC 2024
On 1/6/2024 7:06 PM, Guy Harris wrote:
> On Jan 6, 2024, at 3:35 PM, Paul Eggert<eggert at cs.ucla.edu> wrote:
>
>> On 2024-01-06 14:51, Guy Harris via tz wrote:
>>
>>> POSIX's time API has no notion of "gmtoff" and "stdoff".
>> Although that's true for current POSIX, POSIX 202x/D3 does have tm_gmtoff in struct tm, as a result of Austin Group Defect 1533<https://austingroupbugs.net/view.php?id=1533> which saw steffen and kre as contributors.
> Yes, I know, I signed up for the list so that I could read the draft.
>
> But there's no tm_stdoff, so it wasn't obvious that "gmtoff" and "stdoff" were referring to values in struct tm, and Brooks Harris said
>
>> I just meant the values of gmtoff and stdoff are adjusted to satisfy Posix-time rather than reflect the values in the source files.
> but tm_gmtoff is currently obviously not "adjusted to satisfy Posix-time" as, when tm_gmtoff was introduced, there was no tm_gmtoff in POSIX or, for that matter, in most UN*Xes (was it in *any* UN*Xes before tzcode put it in?), and it's also not "adjusted", it's just the current offset as calculated from the values in the source files.
>
> And, as there's no "tm_stdoff" value even now, there's nothing *to* adjust about stdoff.
Right, but I think there should be. Posix cannot distinguish an stdoff
shift independent from a gmtoff shift.
For example, there is a STDOFF shift at 1971 Oct 31 2:00u from 1:00 to
0:00 (a West shift) in London:
# Zone NAME STDOFF RULES FORMAT [UNTIL]
Zone Europe/London -0:01:15 - LMT 1847 Dec 1 0:00s
0:00 GB-Eire %s 1968 Oct 27
1:00 - BST 1971 Oct 31 2:00u
0:00 GB-Eire %s 1996
0:00 EU GMT/BST
Zic and TzIf reflect this change as a shift in gmtoff, not stdoff:
57722399 1971-10-31 02:59:59 isdst 0 gmtoff 3600 stdoff 0 BST
57722400 1971-10-31 02:00:00 isdst 0 gmtoff 0 stdoff 0 GMT
That's what I mean by "adjusted" for Posix sake. It gives the proper UTC
offset, yes, but not for the right reason. The underlying reason was an
STDOFF shift, presumably stated in the law behind it.
TzDb source has been careful to try to honor the laws and customs behind
the local time rules, and I strongly support that policy. I would
presume this custom originates with the USA laws that specifically
define the time zones as offsets from UTC and the optional one-hour DST
shifts. These are the familiar behaviors of time zones where DST is
"observed".
But many time zones do not follow these familiar patterns and we have to
be careful about that. Many zones have shifted their STDOFF, not always
by an hour, and made multiple DST shifts (co-called "double summertime")
and not always by an hour, and sometimes simultaneously. This can get
tricky.
---------------------
I'm involved with several standards projects at the Society of Motion
Picture and Television Engineers (SMPTE). The television and broadcast
industries have used "SMPTE Timecode" (defined primarily by a standard
named ST12-1) for decades, since the mid 1970s. SMPTE Timecode is used
in very many ways, from labeling of video frames to synchronization and
essentially glues the timekeeping of the whole industry together.
SMPTE Timecode was derived from IRIG and is quite similar to the formats
used by WWV radio and others with addition of video-related metadata.
Timecode is used all over the world, where various video rates are used.
For example most of Europe uses 1/25 (25 frames-per-second). These
systems have an exact relation to running time in seconds and are
sometimes called "integral rate". But the USA, Canada, Japan and others
use the 'strange' rate off 30000/1001, called a "non-integral rate".
This has all sorts of algorithmic ramifications and implementations must
be careful with their math. But this is well-know in the industry.
Meantime there is the matter of audio synchronization of various
frequencies with video. A typical audio frequency is 48000/1.
If you're interested in more detail about video and audio rates and
SMPTE Timecode you might read:
Conversion of Audio Samples to Video Frames
Brooks Harris, May 8, 1998
http://edlmax.com/AudioNTSC.htm
Conversion between SMPTE hh:mm:ss:ff Time Code and Frames
Brooks Harris, EdlMax, LLC. Version V4 2015-04-04
http://edlmax.com/SMPTETimeCodeConversion.htm
In broadcast television it's important to have the hh:mm:ss portion of
the timecode run as close as possible to local time. But SMPTE timecode
and equipment that depends on it cannot tolerate a discontinuity in the
hh:mm:ss:video-frame sequence. To work around this limitation the
industry has typically adopted a procedure called "Daily Jam". Here, DST
shifts, STDOFF shifts and leap-second shifts are deferred to some local
time and the hh:mm:ss:video-frame values are reset ("jammed") to match
the local wall-clock time. This is an imperfect solution which can
produce a "glitch" in the hh:mm:ss:video-frame sequence, especially for
systems running at "non-integer" rates. Therefore the "Daily Jam" is
typically instituted at some local time least likely to disrupt normal
operations. In the USA this time is typically 02:00:00 (2am). Daily Jam
is imperfect but an effective work-around that has been used for decades.
In recent years SMPTE has developed several standards related to
packet-based network "streaming" of video and audio. They've elected to
use IEEE Precision Time Protocol as the primary synchronization
reference. The standards called ST2059-1 and ST2059-2 in particular
define the relation of timecode to PTP time. These also codify the use
of Daily Jam procedures.
User requirements call for use of local time together with video frame
and audio-sample accuracy. And it is this set of challenges that have
brought me (kicking and screaming :-) ) into the wider world of
timekeeping and so to Tz Database.
In 2019 I was invited to make a presentation to ION/PTTI:
Accurate Local Timestamps, Brooks Harris
https://www.ion.org/publications/abstract.cfm?articleID=16763
In this I pointed out that typical timestamps were insufficient to
represent truly unambiguous local time. If we have two ISO 8601
timestamps with date, time and UTC offset:
2023-07-01 00:01:23 -7:00
2023-07-01 00:04:56 -7:00
We can normalize to UTC and determine that the first precedes the
second; precedence is established. Great! This, of course, is the first
of most important aspect of timekeeping. This was the whole point to
Posix-time to begin with; maintain event precedence within the system.
But this entirely misses the fact that the two timestamps may have come
from two different time zones. If you add the time zone it becomes much
clearer what the meaning of the two events are:
2023-07-01 00:01:23 -7:00 America/Los_Angeles
2023-07-01 00:04:56 -7:00 America/Phoenix
But this does not signal that DST was in effect in America/Los_Angeles
and not observed in America/Phoenix. You could add an "isdst" flag:
2023-07-01 00:01:23 -7:00 America/Los_Angeles DST
2023-07-01 00:04:56 -7:00 America/Phoenix STD
That works (more-or-less) in these two typical and familiar cases.
But when you get to other time zones that simple logic may not hold up.
My favorite "kryptonite" example is Europe/Moscow:
# Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S
Rule Russia 1917 only - Jul 1 23:00 1:00 MST #
Moscow Summer Time
Rule Russia 1917 only - Dec 28 0:00 0 MMT #
Moscow Mean Time
Rule Russia 1918 only - May 31 22:00 2:00 MDST #
Moscow Double Summer Time
Rule Russia 1918 only - Sep 16 1:00 1:00 MST
Rule Russia 1919 only - May 31 23:00 2:00 MDST
Rule Russia 1919 only - Jul 1 0:00u 1:00 MSD
Rule Russia 1919 only - Aug 16 0:00 0 MSK
Rule Russia 1921 only - Feb 14 23:00 1:00 MSD
Rule Russia 1921 only - Mar 20 23:00 2:00 +05
Rule Russia 1921 only - Sep 1 0:00 1:00 MSD
Rule Russia 1921 only - Oct 1 0:00 0 -
Rule Russia 1981 1984 - Apr 1 0:00 1:00 S
Rule Russia 1981 1983 - Oct 1 0:00 0 -
Rule Russia 1984 1995 - Sep lastSun 2:00s 0 -
Rule Russia 1985 2010 - Mar lastSun 2:00s 1:00 S
Rule Russia 1996 2010 - Oct lastSun 2:00s 0 -
# Zone NAME STDOFF RULES FORMAT [UNTIL]
Zone Europe/Moscow 2:30:17 - LMT 1880
2:30:17 - MMT 1916 Jul 3 # Moscow Mean Time
2:31:19 Russia %s 1919 Jul 1 0:00u
3:00 Russia %s 1921 Oct
3:00 Russia MSK/MSD 1922 Oct
2:00 - EET 1930 Jun 21
3:00 Russia MSK/MSD 1991 Mar 31 2:00s
2:00 Russia EE%sT 1992 Jan 19 2:00s
3:00 Russia MSK/MSD 2011 Mar 27 2:00s
4:00 - MSK 2014 Oct 26 2:00s
3:00 - MSK
Whoa man!
Take the transition at 1991-03-31 02:00:00
3:00 Russia MSK/MSD 1991 Mar 31 2:00s
2:00 Russia EE%sT 1992 Jan 19 2:00s
This is an STDOFF shift with a simultaneous offsetting DST shift. As
Time And Date puts it:
Mar 31 No change, Mar 31, 1991 - Daylight Saving Time Started
DST started on Sunday, March 31, 1991, 2:00:00 am. However, clocks were
not changed because Moscow switched time zones at the same time.
https://www.timeanddate.com/time/change/russia/moscow?year=1991
This is a "no-op" transition (the YMDhms sequence is not interrupted)
but several important facts changed; both STDOFF and DST shifted, and
the Abbreviation changed. Zic also sets the isdst flag:
670373999 1991-03-31 01:59:59 isdst 0 gmtoff 10800 stdoff 10800 MSK
670374000 1991-03-31 02:00:00 isdst 1 gmtoff 10800 stdoff 10800 EEST
There is another objective in my work. In video we have always been able
to represent any time-point within the 24-hour range, important for
editorial adjustments a duration calculation. To do this comprehensively
with local time you need to know that this day includes a transition and
what and when that transition occurs. Thus, my formats include
information to signal "today is a transition day", "what the transition
is", and "when it occurs". In this Moscow example both STDOFF and DST
shift and these are independently indicated.
D1991-03-30T23:59:59U+03Zeurope/moscowAmskV2021aL16MuX UTC 00670366815
D1991-03-31T00:00:00U+03w01+02Zeurope/moscowAmskV2021aL16S00t01a02cMuX
UTC 00670366816
D1991-03-31T01:59:59U+03w01+02Zeurope/moscowAmskV2021aL16S00t01a02cMuX
UTC 00670374015
D1991-03-31T02:00:00U+03w01+02Zeurope/moscowAeestV2021aL16S01t01a02cMuX
UTC 00670374016
D1991-03-31T23:59:59U+03w01+02Zeurope/moscowAeestV2021aL16S01t01a02cMuX
UTC 00670453215
D1991-04-01T00:00:00U+03Zeurope/moscowAeestV2021aL16S01cMuX UTC 00670453216
There are many details to the formats and implementation I'm working on
but I hope this gives an idea why I believe it is useful to include the
STDOFF data in any timestamp format and to include such transitions in
the output of zic. I also hope Posix might consider adding "tm_stdoff"
to struct tm.
Thanks,
-Brooks
More information about the tz
mailing list