Add support for width-aware noon and midnight options for DayPeriod (#…

…444)

* Add support for NoonMidnight dayPeriods in test data.

- Updates all of the JSON test data to contain fields for noon and midnight dayperiods.

* Add DayPeriod symbols for NoonMidnight

- Adds optional `DayPeriod` symbol for `noon`.
- Adds optional `DayPeriod` symbol for `midnight`.

* Add tests for DayPeirod AmPm and NoonMidnight patterns

- Restructures `format` module to use directory structure instead of single file.
- Adds a `format/tests` directory.
- Moves existing local tests from `format.rs` to `format/tests/mod.rs`
- Adds JSON test data for `DayPeriod` patterns.
- Adds JSON-serializable structs for testing formatting patterns.
- Adds test cases for `DayPeriod` patterns.
- Adds parsing test cases for the `b` `DayPeriod` pattern.

* Remove logic that handles 24:00

- 24:00 will not be a valid time once #355 is fixed.
- Removes logic than handles 24:00 for now.

* Refactor DayPeriod patterns tests to integration tests

- Converts `DayPeriod` pattern tests to be integration tests.
  - Tests no longer direclty use the private `write_pattern()`.
  - Tests now mutate the `DatesV1` struct to the desired pattern,
    using `DateTimeFormat` to format the custom patterns.

* Rewrite symbols!() macro to support serde_none seralization for Options

- Rewrites the `symbols!()` macro as a token tree muncher.
  - For `Option` members, adds serde attributes to skip serializing if none.
  - Otherwise, includes them in the seralization.

* Regenerate test data with optional serializtion for dayperiods

- Regenerates the test data now that `noon` and `midnight` are skipped if not present.
  - Previously `noon` and `midnight` would show up as `null`.

* Minor Test Cleanup

- Moves a few expressions in the dayperiod patterns test to outer loops.

* Make NoonMidnight dependent on granularity of pattern's time.

- Adds capability for a pattern to compute its most granular time.
  - e.g. `h:mm:ss` is `Seconds`.
  - e.g. `h` is `Hours`.
  - e.g. `E, dd/MM/y` is `None`.
- Patterns containing `b` the `NoonMidnight` pattern item will now
  display noon or midnight only if the displayed time falls on the hour.
  - This means that `12:00:00` is always noon-compatible.
  - However, `12:05:15` is noon-compatible only if the display pattern
    does not contain the minutes or seconds.

* Move time granularity functions to format-local helper functions.

- Time granularity functionality is no longer associated with Pattern or
  PatternItem. It is now local to the format module alone as standalone
  functions.

* Move format/mod.rs to format.rs

- Format no longer needs to be a directory.

* Fix access specifiers

- Makes TimeGranularity private instead of public.

* Add minor DayPeriod formatting optimization.

- Only calculates the time granularity if the `DayPeriod` is `NoonMidnight`.

* Cache time granularity on Pattern

- Converts `Pattern` from a tuple struct to a traditional struct.
- Adds a new data member `time_granularity` to `Pattern`.
  - `time_granularity` is a lazily initialized, interrior-mutable cached value.
- Makes `Pattern`'s data members private.
  - The cached `time_granularity` is dependent on the `Pattern`'s `items`.
    It is no longer safe to allow `items` to be publicly accessible,
    because mutating `items` must invalidate the cached granularity.
- Adds new method `items()` to `Pattern` to return a slice of its items.
- Implement `From<Vec<PatternItem>` for `Pattern`
  - This is out of convenience in many places where tuple-struct syntax
    was used previously.

* Clean up Pattern::from_iter

- Pattern::from_iter now uses Pattern::from::<Vec<_>>

* Eagerly evaluate Pattern time granularity

- `Pattern`'s time granularity is no longer lazily evaluated.
- It is instead evaulated on construction.

* Use filter_map instead of flat_map

- filter_map is more specialized, and arguably more readable.

components/datetime/src/fields/symbols.rs

@@ -190,13 +190,15 @@ impl From<Weekday> for FieldSymbol {
 #[derive(Debug, PartialEq, Clone, Copy)]
 pub enum DayPeriod {
     AmPm,
+    NoonMidnight,
 }
 
 impl TryFrom<u8> for DayPeriod {
     type Error = SymbolError;
     fn try_from(b: u8) -> Result<Self, Self::Error> {
         match b {
             b'a' => Ok(Self::AmPm),
+            b'b' => Ok(Self::NoonMidnight),
             b => Err(SymbolError::Unknown(b)),
         }
     }

components/datetime/src/format.rs

@@ -1,12 +1,13 @@
 // This file is part of ICU4X. For terms of use, please see the file
 // called LICENSE at the top level of the ICU4X source tree
 // (online at: https://github.com/unicode-org/icu4x/blob/master/LICENSE ).
+
 use crate::date::{self, DateTimeType};
-use crate::error::DateTimeFormatError;
 use crate::fields::{self, FieldLength, FieldSymbol};
 use crate::pattern::{Pattern, PatternItem};
 use crate::provider;
 use crate::provider::helpers::DateTimeDates;
+use crate::{error::DateTimeFormatError, pattern::TimeGranularity};
 use std::fmt;
 use writeable::Writeable;
 
@@ -96,6 +97,20 @@ fn get_day_of_week(year: usize, month: date::Month, day: date::Day) -> date::Wee
     date::WeekDay::new_unchecked(result as u8)
 }
 
+/// Returns `true` if the most granular time being displayed will align with
+/// the top of the hour, otherwise returns `false`.
+/// e.g. `12:00:00` is at the top of the hour for hours, minutes, and seconds.
+/// e.g. `12:00:05` is only at the top of the hour if the seconds are not displayed.
+fn is_top_of_hour<T: DateTimeType>(pattern: &Pattern, date_time: &T) -> bool {
+    match pattern.most_granular_time() {
+        None | Some(TimeGranularity::Hours) => true,
+        Some(TimeGranularity::Minutes) => u8::from(date_time.minute()) == 0,
+        Some(TimeGranularity::Seconds) => {
+            u8::from(date_time.minute()) + u8::from(date_time.second()) == 0
+        }
+    }
+}
+
 pub fn write_pattern<T, W>(
     pattern: &crate::pattern::Pattern,
     data: &provider::gregory::DatesV1,
@@ -106,7 +121,7 @@ where
     T: DateTimeType,
     W: fmt::Write + ?Sized,
 {
-    for item in &pattern.0 {
+    for item in pattern.items() {
         match item {
             PatternItem::Field(field) => match field.symbol {
                 FieldSymbol::Year(..) => format_number(w, date_time.year(), field.length)?,
@@ -154,15 +169,17 @@ where
                 FieldSymbol::Second(..) => {
                     format_number(w, date_time.second().into(), field.length)?
                 }
-                FieldSymbol::DayPeriod(period) => match period {
-                    fields::DayPeriod::AmPm => {
-                        let symbol =
-                            data.get_symbol_for_day_period(period, field.length, date_time.hour());
-                        w.write_str(symbol)?
-                    }
-                },
+                FieldSymbol::DayPeriod(period) => {
+                    let symbol = data.get_symbol_for_day_period(
+                        period,
+                        field.length,
+                        date_time.hour(),
+                        is_top_of_hour(&pattern, date_time),
+                    );
+                    w.write_str(symbol)?
+                }
             },
-            PatternItem::Literal(l) => w.write_str(l)?,
+            PatternItem::Literal(l) => w.write_str(&l)?,
         }
     }
     Ok(())
@@ -173,7 +190,7 @@ mod tests {
     use super::*;
 
     #[test]
-    fn test_format_numer() {
+    fn test_format_number() {
         let values = &[2, 20, 201, 2017, 20173];
         let samples = &[
             (FieldLength::One, ["2", "20", "201", "2017", "20173"]),

components/datetime/src/pattern/mod.rs

@@ -48,25 +48,67 @@ impl<'p> From<String> for PatternItem {
     }
 }
 
+/// The granularity of time represented in a pattern item.
+/// Ordered from least granular to most granular for comparsion.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
+pub(super) enum TimeGranularity {
+    Hours,
+    Minutes,
+    Seconds,
+}
+
 #[derive(Default, Debug, Clone, PartialEq)]
-pub struct Pattern(pub Vec<PatternItem>);
+pub struct Pattern {
+    items: Vec<PatternItem>,
+    time_granularity: Option<TimeGranularity>,
+}
+
+/// Retrieves the granularity of time represented by a `PatternItem`.
+/// If the `PatternItem` is not time-related, returns `None`.
+fn get_time_granularity(item: &PatternItem) -> Option<TimeGranularity> {
+    match item {
+        PatternItem::Field(field) => match field.symbol {
+            fields::FieldSymbol::Hour(_) => Some(TimeGranularity::Hours),
+            fields::FieldSymbol::Minute => Some(TimeGranularity::Minutes),
+            fields::FieldSymbol::Second(_) => Some(TimeGranularity::Seconds),
+            _ => None,
+        },
+        _ => None,
+    }
+}
 
 impl Pattern {
+    pub fn items(&self) -> &[PatternItem] {
+        &self.items
+    }
+
     pub fn from_bytes(input: &str) -> Result<Self, Error> {
-        Parser::new(input).parse().map(Self)
+        Parser::new(input).parse().map(Pattern::from)
     }
 
     // TODO(#277): This should be turned into a utility for all ICU4X.
     pub fn from_bytes_combination(input: &str, date: Self, time: Self) -> Result<Self, Error> {
         Parser::new(input)
             .parse_placeholders(vec![time, date])
-            .map(Self)
+            .map(Pattern::from)
+    }
+
+    pub(super) fn most_granular_time(&self) -> Option<TimeGranularity> {
+        self.time_granularity
+    }
+}
+
+impl From<Vec<PatternItem>> for Pattern {
+    fn from(items: Vec<PatternItem>) -> Self {
+        Self {
+            time_granularity: items.iter().filter_map(get_time_granularity).max(),
+            items,
+        }
     }
 }
 
 impl FromIterator<PatternItem> for Pattern {
     fn from_iter<I: IntoIterator<Item = PatternItem>>(iter: I) -> Self {
-        let items: Vec<PatternItem> = iter.into_iter().collect();
-        Self(items)
+        Self::from(iter.into_iter().collect::<Vec<_>>())
     }
 }

components/datetime/src/pattern/parser.rs

@@ -155,7 +155,7 @@ impl<'p> Parser<'p> {
                     let replacement = replacements
                         .get_mut(idx as usize)
                         .ok_or(Error::UnknownSubstitution(ch))?;
-                    result.append(&mut replacement.0);
+                    result.extend_from_slice(replacement.items());
                     let ch = chars.next().ok_or(Error::UnclosedPlaceholder)?;
                     if ch != '}' {
                         return Err(Error::UnclosedPlaceholder);
@@ -328,6 +328,14 @@ mod tests {
                     (fields::DayPeriod::AmPm.into(), FieldLength::One).into(),
                 ],
             ),
+            (
+                "hh''b",
+                vec![
+                    (fields::Hour::H12.into(), FieldLength::TwoDigit).into(),
+                    "'".into(),
+                    (fields::DayPeriod::NoonMidnight.into(), FieldLength::One).into(),
+                ],
+            ),
             (
                 "y'My'M",
                 vec![
@@ -353,6 +361,14 @@ mod tests {
                     (fields::DayPeriod::AmPm.into(), FieldLength::One).into(),
                 ],
             ),
+            (
+                "hh 'o''clock' b",
+                vec![
+                    (fields::Hour::H12.into(), FieldLength::TwoDigit).into(),
+                    " o'clock ".into(),
+                    (fields::DayPeriod::NoonMidnight.into(), FieldLength::One).into(),
+                ],
+            ),
             (
                 "hh''a",
                 vec![
@@ -361,6 +377,14 @@ mod tests {
                     (fields::DayPeriod::AmPm.into(), FieldLength::One).into(),
                 ],
             ),
+            (
+                "hh''b",
+                vec![
+                    (fields::Hour::H12.into(), FieldLength::TwoDigit).into(),
+                    "'".into(),
+                    (fields::DayPeriod::NoonMidnight.into(), FieldLength::One).into(),
+                ],
+            ),
         ];
 
         for (string, pattern) in samples {
@@ -385,25 +409,41 @@ mod tests {
     #[test]
     fn pattern_parse_placeholders() {
         let samples = vec![
-            ("{0}", vec![Pattern(vec!["ONE".into()])], vec!["ONE".into()]),
+            (
+                "{0}",
+                vec![Pattern::from(vec!["ONE".into()])],
+                vec!["ONE".into()],
+            ),
             (
                 "{0}{1}",
-                vec![Pattern(vec!["ONE".into()]), Pattern(vec!["TWO".into()])],
+                vec![
+                    Pattern::from(vec!["ONE".into()]),
+                    Pattern::from(vec!["TWO".into()]),
+                ],
                 vec!["ONE".into(), "TWO".into()],
             ),
             (
                 "{0} 'at' {1}",
-                vec![Pattern(vec!["ONE".into()]), Pattern(vec!["TWO".into()])],
+                vec![
+                    Pattern::from(vec!["ONE".into()]),
+                    Pattern::from(vec!["TWO".into()]),
+                ],
                 vec!["ONE".into(), " at ".into(), "TWO".into()],
             ),
             (
                 "{0}'at'{1}",
-                vec![Pattern(vec!["ONE".into()]), Pattern(vec!["TWO".into()])],
+                vec![
+                    Pattern::from(vec!["ONE".into()]),
+                    Pattern::from(vec!["TWO".into()]),
+                ],
                 vec!["ONE".into(), "at".into(), "TWO".into()],
             ),
             (
                 "'{0}' 'at' '{1}'",
-                vec![Pattern(vec!["ONE".into()]), Pattern(vec!["TWO".into()])],
+                vec![
+                    Pattern::from(vec!["ONE".into()]),
+                    Pattern::from(vec!["TWO".into()]),
+                ],
                 vec!["{0} at {1}".into()],
             ),
         ];
@@ -421,10 +461,22 @@ mod tests {
             ("{0}", vec![], Error::UnknownSubstitution('0')),
             ("{a}", vec![], Error::UnknownSubstitution('a')),
             ("{", vec![], Error::UnclosedPlaceholder),
-            ("{0", vec![Pattern(vec![])], Error::UnclosedPlaceholder),
-            ("{01", vec![Pattern(vec![])], Error::UnclosedPlaceholder),
-            ("{00}", vec![Pattern(vec![])], Error::UnclosedPlaceholder),
-            ("'{00}", vec![Pattern(vec![])], Error::UnclosedLiteral),
+            (
+                "{0",
+                vec![Pattern::from(vec![])],
+                Error::UnclosedPlaceholder,
+            ),
+            (
+                "{01",
+                vec![Pattern::from(vec![])],
+                Error::UnclosedPlaceholder,
+            ),
+            (
+                "{00}",
+                vec![Pattern::from(vec![])],
+                Error::UnclosedPlaceholder,
+            ),
+            ("'{00}", vec![Pattern::from(vec![])], Error::UnclosedLiteral),
         ];
 
         for (string, replacements, error) in broken {

components/datetime/src/provider/helpers.rs

@@ -39,6 +39,7 @@ pub trait DateTimeDates {
         day_period: fields::DayPeriod,
         length: fields::FieldLength,
         hour: date::Hour,
+        is_top_of_hour: bool,
     ) -> &Cow<str>;
 }
 
@@ -180,22 +181,20 @@ impl DateTimeDates for provider::gregory::DatesV1 {
         day_period: fields::DayPeriod,
         length: fields::FieldLength,
         hour: date::Hour,
+        is_top_of_hour: bool,
     ) -> &Cow<str> {
-        let widths = match day_period {
-            fields::DayPeriod::AmPm => &self.symbols.day_periods.format,
-        };
+        use fields::{DayPeriod::NoonMidnight, FieldLength};
+        let widths = &self.symbols.day_periods.format;
         let symbols = match length {
-            fields::FieldLength::Wide => &widths.wide,
-            fields::FieldLength::Narrow => &widths.narrow,
+            FieldLength::Wide => &widths.wide,
+            FieldLength::Narrow => &widths.narrow,
             _ => &widths.abbreviated,
         };
-
-        //TODO: Once we have more dayperiod types, we'll need to handle
-        //      this logic in the right location.
-        if u8::from(hour) < 12 {
-            &symbols.am
-        } else {
-            &symbols.pm
+        match (day_period, u8::from(hour), is_top_of_hour) {
+            (NoonMidnight, 00, true) => symbols.midnight.as_ref().unwrap_or(&symbols.am),
+            (NoonMidnight, 12, true) => symbols.noon.as_ref().unwrap_or(&symbols.pm),
+            (_, hour, _) if hour < 12 => &symbols.am,
+            _ => &symbols.pm,
         }
     }
 }