Story: Perform Twisted ticket migration

[danuker]

Migrate the Twisted tickets:

Latest benchmark if nothing goes wrong: Importing the 8 sample tickets took 3 minutes (2.666 tickets/minute; all Twisted tickets would take 64.66 hours if the sample is representative)

Script modification:

• Handle attachments if needed by Twisted Trac #14
• Think about labels: only a few pre-approved ones, instead of all Trac keywords?
• Link to GitHub branch if the field exists; remove the branches/ prefix from SVN
• Re-check before final migration:
  • https://github.com/twisted/twisted-trac-migration/issues/1
  • Write down at least the org members and top 100 users: https://github.com/twisted/twisted-trac-migration/issues/2
• Preserve all trac ticket metadata in machine readable form #17

Migration:

twisted-trac-staging

• generate placeholder tickets until reaching the same ID
• import twisted-trac-staging tickets

twisted/twisted

• If there have been previous partial migrations:
  • Get the latest milestones_created.tsv
  • Get the latest tickets_created.tsv
  • Get the one and only tickets_expected_gold.tsv (generated from a dry-run for all repos at the beginning, to allow ID preservation for linking)
• Use proper credentials (twisted-trac) and other settings in config.py
• Uncomment Twisted contributors from ASSIGNABLE_USERS in config.py
• Disable pre-commit
• Make sure https://github.com/twisted/twisted/settings/key_links has trac# prefix going to https://twistedmatrix.com/trac/ticket/<num>
• Disable Trac edits
• Get latest trac.db using postgres-to-sqlite.sh: check if have latest events in timeline
• Get latest Trac attachments, publish them to https://github.com/twisted/twistedmatrix.com-trac-attachments
• Dry-run
  • check assigned issue numbers (compare fresh tickets_expected.tsv with tickets_expected_gold.tsv)

• Update the plan to limit interfaction only to colaborators "Limit to repository collaborators" - https://github.com/twisted/twisted/settings/interaction_limits
• Update the Twisted PR and issue templates to ask the contributor to wait for the migration to be complete, since opening a PR/issue will cause a ticket in migration to have a misaligned ID.

• Adjust select_tickets to only migrate a single ticket, see if all is all right. Then migrate the next 5 tickets, see if everything is still OK.
• Then migrate the rest. If there are any crashes, check the tickets around the crash (N-1, N, N+1), fix them if needed, and adjust tickets_created.tsv to reflect reality. The migration took 17h14m for https://github.com/twisted/twisted-trac-migration-3 .
• Re-enable pre-commit (.pre-commit-config.yaml as well as org settings)
• Use tickets_created.tsv for redirections to GitHub via JS - done
• Reopen milestones which were open in Trac (script creates everything as closed).
• Remove labels unused for at least 3 open tickets - I kept the "status" labels, for fixed vs. invalid vs. duplicate and so on.
• Push newest attachments; example missing: https://github.com/twisted/twisted-trac-migration/issues/10337
• Restore PR/issue templates 11561 Revert templates and update PR template. twisted/twisted#11562
• delete https://github.com/twisted/twisted-trac-migration-3 and co

• Fix ticket links of the form ticket:1234 in the case of tickets with a different ID on GitHub versus Trac.

From Adi:

• Migrate wiki and host it somewhere Migrate wiki pages to RST and integrate with front page twisted/twisted.github.io#10
  • Done here: https://github.com/twisted/trac-wiki-archive
• Migrate Trac ticket reports ... primarily https://twisted.reviews/ - maybe have that site hosted via GitHub pages
• Ping Glyph to redirect https://twisted.reviews/ to https://github.com/twisted/twisted/issues?q=is%3Apr+is%3Aopen+review%3Arequired - tracked in GitHub based review workflow twisted/twisted#11564
• Manually convert all "review" labels to native github request review from twisted-contributors team
• Also migrate milesone description - can be done manually . ex https://github.com/twisted/twisted-trac-migration/milestone/53 and https://twistedmatrix.com/trac/milestone/Eliminate%20ResourceWarnings

Hooks server (unrelated to migration):

• Twisted hook requirements github-hooks-server#33
• Report known errors via issue/PR comment github-hooks-server#36

[adiroiban]

The migration should start by disabling trac edits for all users other than the person doing the migration.
Trac read-only can continue to be active

Go to https://trac.chevah.com/admin/general/perm and make sure to keep trac-admin only to yourself

---

Also, I think the step would be to create the config first :)

Before the migration we will make sure that both Trac and Github are not updated

The webhook for githhub automation is disabled from here https://github.com/organizations/chevah/settings/hooks/5189828

[adiroiban]

@danuker can this be closed?

[danuker]

No, this is a "story" ticket keeping track of progress (so far, none).

[adiroiban]

Ok. Np.

[adiroiban]

The following Trac ticket metadata should be converted into labels

• status - closed | fixed | new | duplicate | re-open ... etc
• type - defect | enhancement | task | release blocker
• priority
• component

Milestone should be assigned to GitHub Milestones

For all the other "free text" metadata we will only have them as machine readable text:

• reporter
• author
• keywords
• cc
• branch

But we should also include status, type, priority...etc as machine readable.