feat: Make celery task protocol configurable via Django setting (#35789)

This will allow us to test protocol 2 in a stage environment before
removing the override to make 2 the default.

We may have seen a bug where something in celery (or an
associated library) was adding headers to a v1 message as
if it were a v2 message, which caused a bug in ddtrace; such
things may become more likely over time as code is written
with the assumption of v2 messages. Moving to v2 will avoid
those issues.

See https://github.com/edx/edx-arch-experiments/issues/800 for further details.

lms/envs/common.py

@@ -2844,6 +2844,15 @@ DEBUG_TOOLBAR_PATCH_SETTINGS = False
 
 ################################# CELERY ######################################
 
+# Until we've tested protocol 2, stay with protocol 1. It should be
+# fine to just switch to protocol 2, since we're well past celery
+# version 3.1.25 (the first version to support it) but we'll want to
+# test this in a stage environment first.
+#
+# - Docs: https://docs.celeryq.dev/en/stable/history/whatsnew-4.0.html#new-task-message-protocol
+# - Ticket: https://github.com/edx/edx-arch-experiments/issues/800
+CELERY_TASK_PROTOCOL = 1
+
 CELERY_IMPORTS = [
     # Since xblock-poll is not a Django app, and XBlocks don't get auto-imported
     # by celery workers, its tasks will not get auto-discovered:

openedx/core/lib/celery/__init__.py

@@ -23,7 +23,6 @@ from celery import Celery
 # lms.celery. See module docstring!
 APP = Celery('proj')
 
-APP.conf.task_protocol = 1
 # Using a string here means the worker will not have to
 # pickle the object when using Windows.
 APP.config_from_object('django.conf:settings')