dengel / dengel-git

Public
Star on GitHubFork on GitHub
Clone

git clone https://git.dengel.dev/dengel/dengel-git.git

Commit bb7110d
//! NMA-1452 (G5 alpha panic fix): the cross-pack object resolver for the
//! SERVER read path.
//!
//! ## The bug this module fixes
//!
//! `rustgit::pack::odb::resolve_oid_in_git_dir` (frozen upstream, spike
//! 01/03 scope) finds which single pack's .idx contains the requested
//! oid, then resolves any delta chain via that pack's own private
//! `resolve_at`. `resolve_at`'s ref-delta branch looks the base oid up in
//! `oid_to_offset`, a map built from that ONE pack's idx entries only,
//! and panics ("nested ref-delta base <oid> not found in base pack") the
//! moment a ref-delta's base lives anywhere else - a different pack, or
//! loose. A repository that has received a push (2+ packs) very commonly
//! has exactly that shape: `git pack-objects` deltifies the pushed
//! objects against whatever is already reachable, including objects
//! sitting in an older, already-promoted pack. Confirmed on the live G5
//! box: base `6cb0f48...` in pack A, the delta referencing it in pushed
//! pack B - `resolve_oid_in_git_dir` only ever searches pack B, so it
//! panics, the server thread dies, and the client sees an empty/short
//! reply. Same trust-boundary lesson as G1/G3 (see lib.rs's doc comment):
//! a rustgit primitive validated against a single-pack corpus panics the
//! instant dengel-git hands it a shape it was never built for. The fix
//! belongs on this side of the boundary, never inside rustgit.
//!
//! ## Design
//!
//! `ObjectStore::open` builds an oid -> (pack, offset) index once, by
//! parsing every pack's .idx in `objects/pack/` a single time (`.idx`
//! files are exactly the sorted-oid lookup structure gitformat-pack.adoc
//! describes for this purpose - parsing all of them once and keeping the
//! resulting map is the "cache the parsed idx list... once per resolve
//! session" instruction, not a new format fact). `ObjectStore::resolve`
//! then looks up a single oid on demand:
//!
//!   1. Loose objects first (`object::loose::read` - G0), same order the
//!      code being replaced already used.
//!   2. Otherwise the indexed pack entry. A non-delta entry resolves
//!      immediately. An ofs-delta's base offset (gitformat-pack.adoc,
//!      "offset encoding") always names a strictly earlier entry in the
//!      SAME pack file, by construction of the format - so recursing by
//!      offset within that one pack can never itself be the cross-pack
//!      bug and needs no store-wide lookup.
//!   3. A ref-delta's base oid, by contrast, is permitted ANYWHERE in the
//!      object store per gitformat-pack.adoc's "Deltified representation"
//!      (nothing in the format ties a ref-delta's base to its own pack -
//!      that restriction was `resolve_at`'s own bug, not a spec rule).
//!      This is the actual fix: a ref-delta's base recurses through
//!      `resolve` again - loose objects and every other pack included -
//!      instead of one pack's own offset map.
//!
//! Delta-chain depth (steps 2/3 combined, however many pack boundaries it
//! crosses) is bounded by `MAX_DELTA_CHAIN`, the same generous cycle-guard
//! value `ingest::decode_pack` already uses for the identical reason: not
//! a spec limit, just a bound that turns a cyclic or pathological chain
//! into an `Err` instead of unbounded recursion.
//!
//! ## Why not `ingest::decode_pack` per lookup
//!
//! `ingest::decode_pack` already resolves ofs- and ref-deltas against an
//! external `base_lookup` without panicking, and was explicitly built for
//! untrusted/cross-pack input - it would also fix this bug. It was not
//! reused directly here because it decodes and resolves EVERY entry in a
//! pack to answer the request; a single treewalk lookup during a clone
//! only needs the one oid's own delta chain, and enumerating whole packs
//! per lookup is exactly the accidental-O(n^2)-over-all-objects shape the
//! task instructions warn against for a repo with real history. This
//! module instead reads one entry at a time (`objectstore_entry`, positioned
//! reads of the entry header and its zlib stream; it replaced rustgit's
//! `pack::read_raw_entry`, which needed the whole pack file in memory, in
//! NMA-2243) and only ever inflates/applies-delta for the objects actually
//! on the requested oid's own chain.
//!
//! ## Trust tier
//!
//! Every pack byte this module reads comes from THIS repository's own
//! `objects/pack/`, which by construction only ever holds bytes that
//! either (a) this server's own pack writer produced, or (b) arrived
//! over the wire and were already fully structurally validated by
//! `ingest::decode_pack` before being promoted there (see
//! `receive/session.rs`'s pipeline). That is the same trusted tier
//! `rustgit::pack::odb::resolve_oid_in_git_dir` itself always operated
//! in (lib.rs's trust-boundary rule: "rustgit's spike modules... run
//! only behind that boundary, on data that already passed ingress
//! checks") - this module does not widen that tier, it only fixes the
//! single-pack restriction inside it. Entry reads return `Err` on a
//! malformed entry (bit rot included) rather than panicking as rustgit's
//! reader did; delta APPLICATION is not reused as-is - it goes
//! through `ingest::apply_delta_checked` (`Err`, never panics) rather
//! than rustgit's unchecked `delta::apply_delta`, closing that one extra
//! panic surface for the same cost as calling it.

use std::cell::RefCell;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::rc::Rc;

use rustgit::pack::idx;
use rustgit::pack::pack::BaseRef;

use crate::ingest::{self, IngestError};
use crate::object::{self, loose, Kind, ObjectError};

/// Same generous cycle-guard bound `ingest::decode_pack` uses for the
/// identical reason (see that module's own doc comment on
/// `MAX_DELTA_CHAIN`) - not a spec limit, just a bound that makes a
/// cyclic or pathological delta chain an `Err` instead of unbounded
/// recursion. Reused verbatim rather than re-derived.
const MAX_DELTA_CHAIN: u32 = 4096;

#[derive(Debug)]
pub enum StoreError {
    Io(std::io::Error),
    Malformed { what: &'static str, detail: String },
}

impl std::fmt::Display for StoreError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            StoreError::Io(e) => write!(f, "io error: {e}"),
            StoreError::Malformed { what, detail } => write!(f, "malformed {what}: {detail}"),
        }
    }
}

impl std::error::Error for StoreError {}

impl From<std::io::Error> for StoreError {
    fn from(e: std::io::Error) -> Self {
        StoreError::Io(e)
    }
}

impl From<ObjectError> for StoreError {
    fn from(e: ObjectError) -> Self {
        StoreError::Malformed {
            what: "loose object",
            detail: e.to_string(),
        }
    }
}

impl From<IngestError> for StoreError {
    fn from(e: IngestError) -> Self {
        StoreError::Malformed {
            what: "delta application",
            detail: e.to_string(),
        }
    }
}

fn hex(bytes: &[u8]) -> String {
    bytes.iter().map(|b| format!("{b:02x}")).collect()
}

/// A resolve session over one repository's object database (loose +
/// every pack under `objects/pack/`). Built once per server operation
/// that needs to resolve oids (a fetch's treewalk, an ls-refs peel pass,
/// a receive-pack thin-pack base lookup - see this module's doc comment)
/// and reused for every lookup in that operation, so the per-pack .idx
/// parse happens once rather than once per oid.
pub struct ObjectStore {
    git_dir: PathBuf,
    /// oid -> (index into `packs`, offset within that pack).
    index: HashMap<String, (usize, u64)>,
    packs: Vec<PathBuf>,
    /// Lazily opened pack files, same length/order as `packs` - opened
    /// only for packs an actual lookup touches. Entries are read with
    /// positioned reads (`objectstore_entry`), never the whole file.
    pack_files: RefCell<Vec<Option<Rc<std::fs::File>>>>,
}

impl ObjectStore {
    /// Scans `git_dir/objects/pack/*.idx` once, building the oid ->
    /// (pack, offset) index this store answers every `resolve` call
    /// from. A repository with no `objects/pack/` directory at all
    /// (freshly initialized, loose objects only) is a valid empty store,
    /// not an error.
    pub fn open(git_dir: &Path) -> Result<Self, StoreError> {
        let mut packs: Vec<PathBuf> = Vec::new();
        let mut index: HashMap<String, (usize, u64)> = HashMap::new();

        let pack_dir = git_dir.join("objects").join("pack");
        let entries = match std::fs::read_dir(&pack_dir) {
            Ok(e) => e,
            Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
                return Ok(ObjectStore {
                    git_dir: git_dir.to_path_buf(),
                    index,
                    packs,
                    pack_files: RefCell::new(Vec::new()),
                });
            }
            Err(e) => return Err(StoreError::Io(e)),
        };

        let mut idx_paths: Vec<PathBuf> = Vec::new();
        for entry in entries {
            let path = entry?.path();
            if path.extension().map(|e| e == "idx").unwrap_or(false) {
                idx_paths.push(path);
            }
        }
        // Sorted for deterministic, directory-order-independent behavior
        // run to run. Not load-bearing for correctness: a well-formed
        // store never has the same oid in two packs (objects are
        // content-addressed), so which pack "wins" a collision never
        // actually happens on real repository state.
        idx_paths.sort();

        for idx_path in idx_paths {
            let parsed = idx::parse(&idx_path).map_err(StoreError::Io)?;
            let pack_index = packs.len();
            packs.push(idx_path.with_extension("pack"));
            for e in parsed.entries {
                index.entry(e.oid).or_insert((pack_index, e.offset));
            }
        }

        let pack_files = RefCell::new((0..packs.len()).map(|_| None).collect());
        Ok(ObjectStore {
            git_dir: git_dir.to_path_buf(),
            index,
            packs,
            pack_files,
        })
    }

    /// Check a receive tip's presence after validated object publication.
    pub(crate) fn contains(&self, oid: &str) -> std::io::Result<bool> {
        if let Some(&(pack, _)) = self.index.get(oid) {
            return self
                .pack_file(pack)
                .map_err(std::io::Error::other)?
                .metadata()
                .map(|metadata| metadata.is_file());
        }
        if !object::is_lower_hex40(oid) {
            return Ok(false);
        }
        let path = self.git_dir.join("objects").join(&oid[..2]).join(&oid[2..]);
        match std::fs::metadata(path) {
            Ok(metadata) => Ok(metadata.is_file()),
            Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
            Err(error) => Err(error),
        }
    }

    /// Resolve `oid` to (kind, content). `Ok(None)` means `oid` exists
    /// nowhere in this store (neither loose nor in any pack) - never a
    /// panic, on any repo state this store was opened against.
    pub fn resolve(&self, oid: &str) -> Result<Option<(Kind, Vec<u8>)>, StoreError> {
        self.resolve_at_depth(oid, 0, false, usize::MAX)
    }

    /// Stream ordinary existing blob leaves to a sink during connectivity.
    /// Thin delta bases still require their content, with the receive cap.
    pub(crate) fn resolve_for_receive(
        &self,
        oid: &str,
        discard_blob: bool,
    ) -> Result<Option<(Kind, Vec<u8>)>, StoreError> {
        self.resolve_at_depth(oid, 0, discard_blob, ingest::MAX_DECOMPRESSED_SIZE)
    }

    fn resolve_at_depth(
        &self,
        oid: &str,
        depth: u32,
        discard_blob: bool,
        loose_max: usize,
    ) -> Result<Option<(Kind, Vec<u8>)>, StoreError> {
        if depth > MAX_DELTA_CHAIN {
            return Err(StoreError::Malformed {
                what: "delta chain depth",
                detail: format!("chain exceeds {MAX_DELTA_CHAIN} hops resolving {oid} (cycle?)"),
            });
        }

        if let Some(found) = loose::read_with_options(&self.git_dir, oid, discard_blob, loose_max)?
        {
            return Ok(Some(found));
        }

        let Some(&(pack_index, offset)) = self.index.get(oid) else {
            return Ok(None);
        };

        let (kind_code, content) =
            self.resolve_entry(pack_index, offset, depth, discard_blob, loose_max)?;
        let kind = object::kind_from_objcode(kind_code).ok_or_else(|| StoreError::Malformed {
            what: "pack entry type",
            detail: format!("object {oid} has unrecognized pack object-type code {kind_code}"),
        })?;
        Ok(Some((kind, content)))
    }

    fn pack_file(&self, pack_index: usize) -> Result<Rc<std::fs::File>, StoreError> {
        if let Some(file) = self.pack_files.borrow()[pack_index].as_ref() {
            return Ok(file.clone());
        }
        let file = Rc::new(std::fs::File::open(&self.packs[pack_index])?);
        self.pack_files.borrow_mut()[pack_index] = Some(file.clone());
        Ok(file)
    }

    /// Resolves the object at `offset` within pack `pack_index`,
    /// applying its delta chain (if any). `depth` is the number of delta
    /// hops already taken to reach this call, threaded through BOTH
    /// same-pack ofs-delta hops and cross-pack ref-delta hops (the
    /// latter via `resolve_at_depth`, not a fresh `resolve` call - a
    /// fresh call would reset the counter and defeat the cycle guard
    /// across a pack-crossing cycle).
    fn resolve_entry(
        &self,
        pack_index: usize,
        offset: u64,
        depth: u32,
        discard_blob: bool,
        loose_max: usize,
    ) -> Result<(u8, Vec<u8>), StoreError> {
        if depth > MAX_DELTA_CHAIN {
            return Err(StoreError::Malformed {
                what: "delta chain depth",
                detail: format!(
                    "chain exceeds {MAX_DELTA_CHAIN} entries at offset {offset} in {:?}",
                    self.packs[pack_index]
                ),
            });
        }

        let file = self.pack_file(pack_index)?;
        // This repository's own on-disk pack (see this module's doc
        // comment). Read entry by entry; a malformed entry is `Err`.
        let raw = crate::objectstore_entry::read_entry(
            &file,
            offset,
            ingest::MAX_DECOMPRESSED_SIZE,
            discard_blob,
        )?;

        match raw.base {
            BaseRef::None => Ok((raw.kind, raw.content)),
            BaseRef::Offset(base_offset) => {
                // gitformat-pack.adoc's ofs-delta offset always names a
                // strictly earlier entry in THIS SAME pack file - never
                // crosses a pack boundary, so this recursion alone can
                // never be the cross-pack bug this module exists to fix.
                let (base_kind, base_content) =
                    self.resolve_entry(pack_index, base_offset, depth + 1, false, loose_max)?;
                let content = ingest::apply_delta_checked(
                    &base_content,
                    &raw.content,
                    ingest::MAX_DECOMPRESSED_SIZE,
                )?;
                Ok((base_kind, content))
            }
            BaseRef::Oid(base_oid_bytes) => {
                // THE FIX: a ref-delta's base may be loose or in ANY
                // other pack. Resolve it through the whole store, not
                // this one pack's offset map (the single-pack
                // restriction that made
                // rustgit::pack::odb::resolve_oid_in_git_dir panic).
                let base_oid = hex(&base_oid_bytes);
                let Some((base_kind, base_content)) =
                    self.resolve_at_depth(&base_oid, depth + 1, false, loose_max)?
                else {
                    return Err(StoreError::Malformed {
                        what: "ref-delta base",
                        detail: format!("base {base_oid} not found anywhere in the object store"),
                    });
                };
                let content = ingest::apply_delta_checked(
                    &base_content,
                    &raw.content,
                    ingest::MAX_DECOMPRESSED_SIZE,
                )?;
                Ok((object::kind_to_objcode(base_kind), content))
            }
        }
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use std::process::Command;

    fn scratch_dir(name: &str) -> PathBuf {
        let dir = std::env::temp_dir().join(format!(
            "dengel-git-objectstore-test-{}-{}-{}",
            name,
            std::process::id(),
            std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .unwrap()
                .as_nanos()
        ));
        std::fs::create_dir_all(&dir).unwrap();
        dir
    }

    fn run_git(dir: &Path, args: &[&str]) -> Vec<u8> {
        let output = Command::new("git")
            .current_dir(dir)
            .args(args)
            .output()
            .unwrap();
        assert!(
            output.status.success(),
            "git {args:?} failed: {}",
            String::from_utf8_lossy(&output.stderr)
        );
        output.stdout
    }

    fn run_git_ok(dir: &Path, args: &[&str]) {
        run_git(dir, args);
    }

    #[test]
    fn empty_repo_has_no_objects_and_no_panic() {
        let git_dir = scratch_dir("empty");
        run_git_ok(&git_dir, &["init", "-q", "--bare", "."]);
        let store = ObjectStore::open(&git_dir).unwrap();
        let bogus = "0000000000000000000000000000000000000000";
        assert!(matches!(store.resolve(bogus), Ok(None)));
    }

    #[test]
    fn resolves_a_loose_object() {
        let git_dir = scratch_dir("loose");
        run_git_ok(&git_dir, &["init", "-q", "--bare", "."]);
        let oid = loose::write(&git_dir, Kind::Blob, b"hello\n").unwrap();
        let store = ObjectStore::open(&git_dir).unwrap();
        let (kind, content) = store.resolve(&oid).unwrap().unwrap();
        assert_eq!(kind, Kind::Blob);
        assert_eq!(content, b"hello\n");
    }

    /// The actual NMA-1452 repro, built with real git rather than a
    /// hand-assembled pack (same "real oracle" discipline as
    /// ingest::decode_pack's own tests): a bare repo whose first pack
    /// (pack A) holds an initial commit's objects, then a second commit
    /// is packed on top into a SECOND pack (pack B) via `git repack -d
    /// --no-write-bitmap-index -q` after a plain `commit` (which by
    /// default writes loose objects) followed by `git pack-objects` on
    /// just the new commit's objects deltified against the old ones -
    /// reproducing exactly the "pushed pack has a ref-delta whose base
    /// is in a different, already-promoted pack" shape the bug report
    /// describes, without going through dengel-git's own receive path
    /// (that end-to-end path is covered separately, over the wire, by
    /// this crate's g2_serve test suite).
    #[test]
    fn resolves_a_ref_delta_whose_base_is_in_a_different_pack() {
        let dir = scratch_dir("cross_pack");
        run_git_ok(&dir, &["init", "-q", "-b", "main"]);
        run_git_ok(
            &dir,
            &["config", "user.email", "objectstore-test@example.com"],
        );
        run_git_ok(&dir, &["config", "user.name", "objectstore-test"]);

        std::fs::write(dir.join("a.txt"), "one\ntwo\nthree\n".repeat(80)).unwrap();
        run_git_ok(&dir, &["add", "a.txt"]);
        run_git_ok(&dir, &["commit", "-q", "-m", "first"]);
        let first_head = String::from_utf8(run_git(&dir, &["rev-parse", "HEAD"]))
            .unwrap()
            .trim()
            .to_string();

        let git_dir = dir.join(".git");
        // Pack A: everything reachable from the first commit, written as
        // an actual pack (not loose) so it lands in objects/pack/ as its
        // own separate .idx/.pack pair.
        let oids_a = run_git(&dir, &["rev-list", "--objects", &first_head]);
        write_pack_from_stdin(&dir, &oids_a, "pack-a");

        // Second commit, edited so pack-objects has a good delta
        // candidate against the first commit's blob.
        std::fs::write(dir.join("a.txt"), "one\ntwo\nTHREE-EDITED\n".repeat(80)).unwrap();
        run_git_ok(&dir, &["add", "-A"]);
        run_git_ok(&dir, &["commit", "-q", "-m", "second"]);
        let second_head = String::from_utf8(run_git(&dir, &["rev-parse", "HEAD"]))
            .unwrap()
            .trim()
            .to_string();

        // Pack B: ONLY the second commit's own new objects
        // (`rev-list first..second`), deltified by real git against
        // whatever it can see reachable in the repo - which, for the
        // edited a.txt blob, is the first commit's a.txt blob sitting in
        // pack A, not anything in pack B itself. This is the exact
        // cross-pack ref-delta shape the bug report confirms on the live
        // box (base in pack A, delta in pushed pack B).
        let oids_b = run_git(
            &dir,
            &[
                "rev-list",
                "--objects",
                &format!("{first_head}..{second_head}"),
            ],
        );
        write_pack_from_stdin(&dir, &oids_b, "pack-b");

        // Sanity: two packs actually landed (otherwise this test isn't
        // exercising the bug at all).
        let pack_count = std::fs::read_dir(git_dir.join("objects/pack"))
            .unwrap()
            .filter(|e| {
                e.as_ref()
                    .unwrap()
                    .path()
                    .extension()
                    .map(|x| x == "pack")
                    .unwrap_or(false)
            })
            .count();
        assert_eq!(
            pack_count, 2,
            "expected exactly two packs (pack A + pack B) for this repro to be meaningful"
        );

        let store = ObjectStore::open(&git_dir).unwrap();

        // Resolve every object reachable from the SECOND commit - this
        // walks straight through the cross-pack ref-delta. Before the
        // fix, this panicked ("nested ref-delta base ... not found in
        // base pack") the moment it reached that entry.
        let all_oids =
            String::from_utf8(run_git(&dir, &["rev-list", "--objects", &second_head])).unwrap();
        for line in all_oids.lines() {
            let oid = line.split(' ').next().unwrap();
            let (kind, content) = store
                .resolve(oid)
                .unwrap()
                .unwrap_or_else(|| panic!("{oid} did not resolve"));
            let kind_name = match kind {
                Kind::Blob => "blob",
                Kind::Tree => "tree",
                Kind::Commit => "commit",
                Kind::Tag => "tag",
            };
            let git_content = run_git(&dir, &["cat-file", kind_name, oid]);
            assert_eq!(content, git_content, "content mismatch for {oid}");
        }
    }

    /// Writes every oid on `oids_stdin` (one per line, "oid path" or bare
    /// oid, as `git rev-list --objects` produces) into a new pack in
    /// `dir`'s `objects/pack/`, via real `git pack-objects`.
    fn write_pack_from_stdin(dir: &Path, oids_stdin: &[u8], label: &str) {
        use std::io::Write;
        let prefix = dir.join(".git/objects/pack").join(label);
        let mut child = Command::new("git")
            .current_dir(dir)
            .args(["pack-objects", "--", prefix.to_str().unwrap()])
            .stdin(std::process::Stdio::piped())
            .stdout(std::process::Stdio::piped())
            .stderr(std::process::Stdio::piped())
            .spawn()
            .expect("failed to spawn git pack-objects");
        child.stdin.take().unwrap().write_all(oids_stdin).unwrap();
        let output = child.wait_with_output().unwrap();
        assert!(
            output.status.success(),
            "git pack-objects failed: {}",
            String::from_utf8_lossy(&output.stderr)
        );
    }

    #[test]
    fn resolves_an_ofs_delta_within_a_single_pack() {
        // git's default packing for a single `pack-objects` invocation
        // over a small linear history commonly produces at least one
        // ofs-delta among the blobs/trees - exercised implicitly by
        // `resolves_a_ref_delta_whose_base_is_in_a_different_pack` above
        // (pack A alone has two blob generations... actually pack A only
        // has the first commit's objects, so this needs its own fixture
        // with delta-worthy history packed together in ONE invocation).
        let dir = scratch_dir("ofs_delta");
        run_git_ok(&dir, &["init", "-q", "-b", "main"]);
        run_git_ok(
            &dir,
            &["config", "user.email", "objectstore-test@example.com"],
        );
        run_git_ok(&dir, &["config", "user.name", "objectstore-test"]);

        std::fs::write(dir.join("a.txt"), "one\ntwo\nthree\n".repeat(80)).unwrap();
        run_git_ok(&dir, &["add", "a.txt"]);
        run_git_ok(&dir, &["commit", "-q", "-m", "first"]);
        std::fs::write(dir.join("a.txt"), "one\ntwo\nTHREE-EDITED\n".repeat(80)).unwrap();
        run_git_ok(&dir, &["add", "-A"]);
        run_git_ok(&dir, &["commit", "-q", "-m", "second"]);
        let head = String::from_utf8(run_git(&dir, &["rev-parse", "HEAD"]))
            .unwrap()
            .trim()
            .to_string();

        let git_dir = dir.join(".git");
        let all_oids = run_git(&dir, &["rev-list", "--objects", &head]);
        write_pack_from_stdin(&dir, &all_oids, "pack-single");

        let store = ObjectStore::open(&git_dir).unwrap();
        let all_oids_text = String::from_utf8(all_oids).unwrap();
        for line in all_oids_text.lines() {
            let oid = line.split(' ').next().unwrap();
            assert!(
                store.resolve(oid).unwrap().is_some(),
                "{oid} did not resolve"
            );
        }
    }
}