Merge branch 'main' into pencil_search

notebooks/join_known_objects_example.ipynb

@@ -0,0 +1,169 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "a89ad362-6ed6-489f-806c-2fd94fc9356d",
+   "metadata": {},
+   "source": [
+    "# Example Known Object Labeling\n",
+    "\n",
+    "This notebook serves as an example (and usable tool) for labeling objects in the results file as corresponding to a known object. It assumes the user has run KBMOD to produce a results .ecsv and has access to a table of known results.\n",
+    "\n",
+    "This notebook uses specific files and parameters from the DEEP reprocessing."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0b1d47da-db6f-4530-93a2-9cb83a6af145",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from astropy.table import Table\n",
+    "import numpy as np\n",
+    "\n",
+    "from kbmod.filters.known_object_filters import KnownObjsMatcher\n",
+    "from kbmod.results import Results"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "43b6aaa7-9bf3-4ed7-8659-8daea4ed4765",
+   "metadata": {},
+   "source": [
+    "We start by loading the results data and known object data. The results data is the ecsv file produced by a KBMOD run and contains information on each trajectory found. The known object table is a given file with information on the location (RA, dec) of each observation at different time steps.\n",
+    "\n",
+    "We also extract the required metadata (global WCS and a list of all observation times) from the results."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6ce8dcad-fa91-4267-b2cf-7c787ac0d665",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# These two files are specific to the UW DEEP reprocessing runs and should be replaced\n",
+    "# by the user's files of interest.\n",
+    "res_file = \"/epyc/projects/kbmod/runs/DEEP/results/20190402_A0b_001.results.ecsv\"\n",
+    "known_file = \"/epyc/projects/kbmod/data/fakes_detections_joined.fits\"\n",
+    "\n",
+    "in_results = Results.read_table(res_file)\n",
+    "print(f\"Loaded a results table with {len(res_file)} entries and columns:\\n{in_results.colnames}\")\n",
+    "\n",
+    "wcs = in_results.wcs\n",
+    "if wcs is None:\n",
+    "    raise ValueError(\"WCS missing from results file.\")\n",
+    "\n",
+    "if \"mjd_mid\" in in_results.table.meta:\n",
+    "    obstimes = np.array(in_results.table.meta[\"mjd_mid\"])\n",
+    "else:\n",
+    "    raise ValueError(\"Metadata 'mjd_mid' missing from results file.\")\n",
+    "print(f\"Loaded {len(obstimes)} timestamps.\")\n",
+    "\n",
+    "known_table = Table.read(known_file)\n",
+    "print(\n",
+    "    f\"\\n\\nLoaded a known objects table with {len(known_table)} entries \"\n",
+    "    f\"and columns:\\n{known_table.colnames}\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c06cc1e2-2186-4418-a746-1bbe5e7bc971",
+   "metadata": {},
+   "source": [
+    "We use the `KnownObjsMatcher` to determine which of the found results correspond to previously known objects. `KnownObjsMatcher` provides the ability to match by either the number or ratio of observations that are in close proximity to the known object. Here we use a minimum number with reasonable proximity thresholds in space and time."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "19eeeb00-b7f2-4969-8737-185f9161b34a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "min_obs = 10\n",
+    "fake_matcher = KnownObjsMatcher(\n",
+    "    known_table,\n",
+    "    obstimes,\n",
+    "    matcher_name=\"known_matcher\",\n",
+    "    sep_thresh=2.0,  # Obs must be within 2 arcsecs.\n",
+    "    time_thresh_s=600.0,  # Obs must be within 10 minutes.\n",
+    "    name_col=\"ORBITID\",  # For the DEEP-data known objects only.\n",
+    ")\n",
+    "\n",
+    "# First create the matches column.\n",
+    "fake_matcher.match(in_results, wcs)\n",
+    "\n",
+    "# Second filter the matches.\n",
+    "fake_matcher.match_on_min_obs(in_results, min_obs)\n",
+    "\n",
+    "matched_col_name = fake_matcher.match_min_obs_col(min_obs)\n",
+    "print(f\"Matches stored in column '{matched_col_name}'\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2d8faadb-a496-4798-a0df-27712e4db8cd",
+   "metadata": {},
+   "source": [
+    "Iterate over the matched column computing a Boolean of whether there was any match (True if the match list is not empty). Add the resulting list as a new \"is_known\" column."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b6490988-242e-4a0f-b355-fcb1804d118d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "is_known = ~in_results.is_empty_value(matched_col_name)\n",
+    "in_results.table[\"is_known\"] = is_known\n",
+    "matched_count = np.count_nonzero(is_known)\n",
+    "\n",
+    "print(f\"Found {matched_count} of the {len(in_results)} results matched known objects.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c4c79978-ed82-4261-abd4-14e745efb09a",
+   "metadata": {},
+   "source": [
+    "We could save the resulting joined table using:\n",
+    "```\n",
+    "in_results.write_table(output_filename)\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "56aadd08-aaed-4de5-a9eb-77ecafef1e55",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Jeremy's KBMOD",
+   "language": "python",
+   "name": "kbmod_jk"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

src/kbmod/filters/__init__.py

@@ -1,5 +1,6 @@
 from . import (
     clustering_filters,
+    known_object_filters,
     sigma_g_filter,
     stamp_filters,
 )