fix storage/quota issue

Author: ameisner

DP1/100_How_to_Use_RSP_Tools/105_Image_reprocessing/105_6_Create_custom_coadd_images.ipynb

@@ -19,7 +19,7 @@
     "Data Release: DP1 <br>\n",
     "Container Size: large <br>\n",
     "LSST Science Pipelines version: r29.1.1 <br>\n",
-    "Last verified to run: 2025-06-24 <br>\n",
+    "Last verified to run: 2025-07-25 <br>\n",
     "Repository: <a href=\"https://github.com/lsst/tutorial-notebooks\">github.com/lsst/tutorial-notebooks</a> <br>"
    ]
   },
@@ -543,7 +543,7 @@
    "id": "85394b0f-7222-4624-89af-4c1f9c70d7d8",
    "metadata": {},
    "source": [
-    "Users cannot write to the main DP1 Butler repository. Instead, make a writable local repository within your current directory where custom coadd outputs will be written. The call to `use_local_butler` creates a new directory that will contain the local Butler repository, within your current working directory."
+    "Users cannot write to the main DP1 Butler repository. Instead, make a writable repository where custom coadd outputs will be written. The call to `use_local_butler` creates a new directory that will contain the local Butler repository, within the `/deleted-sundays` temporary space that gets deleted weekly. Use this temporary space because the custom coadd processing produces of order a gigabyte of intermediate files."
    ]
   },
   {
@@ -553,7 +553,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "local_repo_name = os.path.join(os.getenv(\"HOME\"), \"my_local_repo\")\n",
+    "local_repo_name = os.path.join('/deleted-sundays', my_username, \"my_local_repo\")\n",
     "out_butler = executor.use_local_butler(local_repo_name)"
    ]
   },
@@ -634,13 +634,86 @@
     "> Figure 2: The custom `deep_coadd` image using only the selected subset of input visit images."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "a95de02e-d676-4ca3-8fb8-d1afe410e1d4",
+   "metadata": {},
+   "source": [
+    "### 3.4. Copy the custom coadd output file"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aa9eacc5-18b0-4a8a-aa05-4b45c12aeee8",
+   "metadata": {},
+   "source": [
+    "The writable Butler repository used for custom coaddition is located within a temporary storage area. The Butler `retrieveArtifacts` method can be used to retrieve a copy of just the custom coadd output file, without copying intermediate files generated by the processing, and placing the custom coadd output file in a desired location. This can be useful to preserve the custom coadd image file without filling up a significant amount of disk quota in a permanent location. Start by retrieving the dataset reference for the custom coadd image output file."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
    "id": "91ee3647-96c9-4aa9-9d61-ced39b8c1990",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "refs = out_butler.query_datasets('deep_coadd_predetection',\n",
+    "                                 collections=executor.quantum_graph.metadata[\"output_run\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f5dee19c-5bc9-4f5b-af9c-aaabfec6ee77",
+   "metadata": {},
+   "source": [
+    "As an example, choose to put the copy of the custom coadd output file in your space within the `/deleted-sundays` area."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9e55d973-a30d-4891-a896-6b62e6b08a9c",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "outdir = os.path.join('/deleted-sundays', my_username)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e639644d-14e2-47ba-92d1-03a952f634c2",
+   "metadata": {},
+   "source": [
+    "*Option*: instead choose to copy the custom coadd output file to your home directory, where it will not be deleted within a week."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e0f81548-e8c0-4854-b2f7-767da392f27b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# outdir = os.getenv(\"HOME\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1fdd59a0-0454-441d-b60f-4d6aeb10d147",
+   "metadata": {},
+   "source": [
+    "Copy the FITS file containing the custom coadd image to the desired location."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a3913c72-355a-4289-8061-b22785ff7df9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "out_butler.retrieveArtifacts(refs, outdir, preserve_path=False, transfer=\"copy\", overwrite=True)"
+   ]
   }
  ],
  "metadata": {