Commit Graph

3988 Commits

Author SHA1 Message Date
Florian Haas
26281cbe36 Fix profile image URLs for image storage on non-public S3 buckets
In image_helpers.py, the _get_profile_image_urls() method would append
"?v=<version>" to the query string for serving profile images.

This might break serving profile images if

* EDXAPP_PROFILE_IMAGE_BACKEND was configured with its class option
  set to django.storages.s3boto3.S3Boto3Storage (or its deprecated
  predecedessor, django.storages.s3boto.S3BotoStorage), and
* that backend used signed URLs with query-string authentication (i.e.
  was *not* configured with an S3 custom domain).

When both the above conditions are met, then the URL returned by the
storage backend's url() method already contains "?", and
_get_profile_image_urls() would add another. This results in a query
string that doesn't exactly violate RFC 3986, but is discouraged by
it.[1]

Amazon S3 itself may be able to parse these query strings correctly,
but other S3 API implementations (such as Ceph radosgw[2]) may not,
and the problem is easily avoided by just looking for "?" in the
rendered URL, and using "&v=<version>" instead if we find a match.

The proper way of appending the v=<version> query parameter would
probably be to pull the URL and the query string apart and then back
together[3], but that's most likely overdoing it.

[1] https://tools.ietf.org/html/rfc3986#section-3.4 says:
"However, as query components are often used to carry identifying
information in the form of "key=value" pairs and one frequently used
value is a reference to another URI, it is sometimes better for
usability to avoid percent- encoding those characters." ("Those
characters" being "/" and "?".)

[2] https://docs.ceph.com/docs/master/radosgw/s3/

[3] https://docs.python.org/3/library/urllib.parse.html
2020-07-06 11:09:16 +02:00
Robert Raposa
7f22041fc1 ARCHBOM-1316: always enable some course_experience flags (#24322)
In order to remove the deprecated flag_undefined_default=True
argument, this commit updates the following flags to always be
enabled using a new temporary class:

- course_experience.course_outline_page
- course_experience.unified_course_tab

Adds a temporary setting `USE_DEFAULT_TRUE_NAMESPACE`,
to enable a monitored rollout of this change.

TNL-7061 is the ticket where these flags will actually be
removed. This requires more careful work including removing
all dead code, and potentially refactoring tests that were
testing shared functionality, but only when the flag was
False.

ARCHBOM-1316
2020-07-02 12:00:23 -04:00
Michael Terry
3030efec78 AA-177: Add masquerading for course home MFE
- Looks at masquerading config for dates, outline, metadata, and
  celebration APIs in course_home_api / courseware_api.
- Consolidates and cleans up places we check whether masquerading
  gives us full access to a course.
2020-06-29 12:37:47 -04:00
Aarif
9e34cb3df9 Removed deprecated future imports (#24295) 2020-06-28 23:40:03 +05:00
Zainab Amir
0b80987ba0 Add vem_enabled_courses_percentage (#24294)
Add a field to VEM config model that will decide the percentage of
courses allowed to go to VEM pipeline. The courses that don't meet the
criteria will go to VEDA.

PROD-1722
2020-06-26 14:30:42 +05:00
Calen Pennington
d933541dad Merge pull request #24302 from cpennington/prioritize-access-denied-messages
Prioritize access denied messages
2020-06-25 10:15:57 -04:00
Calen Pennington
aef3a20e5d Prioritize access denials in block transformers from groups that have messages associated with them 2020-06-25 09:51:50 -04:00
Kyle McCormick
c3d7ab9943 Fix programs cache ADR formatting (#24320) 2020-06-24 11:55:43 -04:00
Nimisha Asthagiri
a219381361 Update 0001-refactor-waffle-flag-default.rst (#24308) 2020-06-23 21:03:06 -04:00
Robert Raposa
f060872878 update waffle flag and docs (#24299)
- update ADR to provide more alternatives for updating
the default value of a flag.
- add a `flag_` prefix to the flag metrics
- add module-level note about flag metrics
- add NewRelic query example and warning
- fix typo in toggle annotation

ARCHBOM-1302
2020-06-23 13:55:06 -04:00
Aarif
27480cdd2f Removed deprecated future imports 2020-06-23 18:51:43 +05:00
Robert Raposa
a1ed944c72 add temp waffle flag default metric (#24284)
The argument flag_undefined_default is soon to be retired
once ARCHBOM-132 is closed. The following will be used to
help ensure the rollout is complete.

- Add a temporary metric if flag_undefined_default is used.
- Add deprecation warning for flag_undefined_default.
- Add minor fix for waffle flag metric when no request found.

ARCHBOM-132
2020-06-22 13:53:17 -04:00
Robert Raposa
5ea9d313ad add ADR for waffle flag default (#24272)
We are changing how we handle updates to a waffle flag
default, and the ADR explains why and how.

ARCHBOM-1303
2020-06-22 10:12:10 -04:00
Robert Raposa
343a5a8a74 update waffle flag custom metrics (#24270)
The previous version of this code used the Django Setting
ENABLE_WAFFLE_FLAG_METRIC to determine whether to add a single
metric with a dict of details about all flags. Due to
NewRelic's 256 character limit on the metric value, this was
getting truncated.

This new version instead uses the Django Setting
WAFFLE_FLAG_CUSTOM_METRICS, a list of waffle flag names to
instrument.

The name of each custom metric will match the name of the flag.
The value of the custom metric could be False, True, or Both.

The value Both would mean that the flag had both a True and False
value at different times during the transaction. This is most
likely due to having a check_before_waffle_callback, as is the
case with CourseWaffleFlag.

ARCHBOM-132
2020-06-22 10:11:40 -04:00
Ahtisham Shahid
5707bbdc90 updated confirm_email field type (#24205)
* updated confirm_email field type and removed confirm email form v1
2020-06-22 17:10:41 +05:00
David Ormsbee
0b4cf7e06c Merge pull request #23039 from open-craft/samuel/fix-unescaped-selector
SE-2176 Fix elem not selected if id contains special chars
2020-06-19 12:13:20 -04:00
Leonardo Martinez
b4fee68283 Fix the DeprecationWarning for unescape (#23936)
This PR solves the DeprecationWarning mentioned in:
https://build.testeng.edx.org/job/edx-platform-python-pipeline-master/warning_5freport_5fall_2ehtml/

HTMLParser was renamed in html.parser in Python3:
https://docs.python.org/2/library/htmlparser.html#module-HTMLParser

* html_parser.HTMLParser().unescape from six.moves has been deprecated

* instead use html.unescape from Python3

Documentation for unescape in Python3:
https://docs.python.org/3/library/html.html#html.unescape

- html_parser from six.moves has been deprecated

- instead use html.parser from Python3

- Order imports using isort

- Delete unused import crum
2020-06-19 09:27:45 -04:00
stvn
994b2b002f Add explicit masquerade support to courseware API 2020-06-18 14:54:49 -07:00
stvn
0ae91c0921 Replace references to 'log.warn' with 'log.warning'
to remove some `DeprecationWarning`s from the logs
as the former method as been deprecated since Python 3.3 [1][2].

- [1] https://github.com/python/cpython/blob/3.3/Lib/logging/__init__.py#L1252-L1253
- [2] https://stackoverflow.com/a/15655674
2020-06-18 12:08:15 -07:00
Robert Raposa
a1572dafce add waffle_flag custom metric (#24244)
If setting ENABLE_WAFFLE_FLAG_METRIC is True, a custom
metric will be added with the values of all WaffleFlag
and CourseWaffleFlags seen during the transaction.

Metric flag values could be False, True, or Both.
The value Both would mean that the flag had both
a True and False value at different times through
the transaction. This is most likely due to having a
check_before_waffle_callback, as is the case with
CourseWaffleFlag.

Example metric value:

    "{'another.course.flag': 'False', 'some.flag': 'False', 'some.course.flag': 'Both'}"

Warning: NewRelic does not recommend large custom
metric values due to the potential performance
impact on the agent, so you may just want to
enable when researching usage of a particular flag.
Metric values longer than 255 are truncated.

TODO: A how_to can be added later if we find this
useful, including helpful querying tips.

ARCHBOM-132
2020-06-18 12:21:57 -04:00
Michael Terry
d325943fe5 Merge pull request #24169 from edx/mikix/celebration
AA-137: Support courseware celebrations
2020-06-18 10:49:36 -04:00
atesker
12aba3bd94 PR comments 2020-06-18 10:31:24 -04:00
atesker
8bf47afdae Initial ADR draft - comments and move 2020-06-18 10:31:24 -04:00
atesker
992ca3174e Initial ADR draft 2020-06-18 10:31:24 -04:00
Ali Akbar
edd9b02d52 Merge pull request #180 from edx/sustaining-xsslint-security-fixes
Sustaining xsslint security fixes
2020-06-18 11:32:39 +05:00
Ahtisham Shahid
1743e0e590 Merge pull request #24228 from edx/ahtisham/PROD-1679
Removed confirm email after SSO
2020-06-17 12:25:26 +05:00
Michael Terry
450072582e AA-137: Support courseware celebrations
- Add a new CourseEnrollmentCelebration model, which ties a
  course enrollment to some booleans about progress celebrations
- Add serialization of the new model to the existing courseware_api
  app's existing course info view
- Add new API in courseware_api to update a celebration model
2020-06-16 15:19:21 -04:00
Ahtisham Shahid
340e00988f Removed confirm email after SSO 2020-06-16 14:06:52 +05:00
SaadYousaf
52cfe647b3 syncing certificates on course update on credential side. 2020-06-16 11:58:49 +05:00
Samuel Walladge
43f0cd72af Merge remote-tracking branch 'origin/master' into samuel/fix-unescaped-selector 2020-06-16 09:52:38 +09:30
Michael Terry
557079042f Guard highlights against null module
Sometimes a course module will not exist and was causing exceptions
in our weekly highlights mail code. This will hopefully guard against
that a bit better.
2020-06-12 14:57:09 -04:00
Calen Pennington
06e5e2db51 Merge pull request #24144 from cpennington/hide-dates-banner-archived
Hide dates banner archived
2020-06-11 14:03:09 -04:00
Calen Pennington
86e2677f26 Use a standard function to find out if a course has ended 2020-06-11 11:40:56 -04:00
Calen Pennington
1cf46e76d0 Don't compare None to a date when a course doesn't have an end date 2020-06-11 11:40:56 -04:00
Stu Young
b370745b9b management command to retire a single user (#24188) 2020-06-10 18:07:25 -04:00
Calen Pennington
c6695e0b6f Only actually reset user schedule if they have missed deadlines and not missed any gated content deadlines 2020-06-10 15:18:53 -04:00
Daniel Francis
4dc3251af5 render_to_response deprecated in Django 3 (#24141)
Fixing RemovedInDjango30Warnings

**Background:** The `django.shortcuts` method `render_to_response` became deprecated in [Django 1.3](https://docs.djangoproject.com/en/3.0/releases/1.3/), when  `render` was introduced.

Per the documentation:

> render() is the same as a call to render_to_response() with a context_instance argument that forces the use of a RequestContext.

Both return an `HttpResponse` object.

**Context:** We changed two statements: An import line and the call to the method, adding explicit parameter names to improve readability.

**Before:**
```
from django.shortcuts import get_object_or_404, render_to_response
...
return render_to_response("teams/teams.html", context)


```

**After**
```
from django.shortcuts import get_object_or_404, render
...
return render(
            request=request,
            template_name="teams/teams.html",
            context=context
        )

```
2020-06-10 14:25:04 -04:00
Ali-D-Akbar
0ecbe873d9 PROD-1574 2020-06-10 21:07:10 +05:00
adeelehsan
d879f9fd00 Merge pull request #23980 from edx/aehsan/PROD-1582/specify_user_message_for_about_me_in_profile
Specify bio field error message for user
2020-06-10 14:46:04 +05:00
adeel khan
76419f9d01 Merge pull request #23913 from edx/adeel/prod_1505_improve_security_lockouts_logic
Improving user locked out logic.
2020-06-10 14:21:16 +05:00
adeelehsan
4e1fe2045f Specify bio message for user
Currently user message is generic. Specifying for
bio field so that it make more sense to user.

PROD-1582
2020-06-10 14:19:39 +05:00
Waheed Ahmed
6b268c37b4 Rate limit logistration endpoints.
PROD-1506
2020-06-10 13:33:26 +05:00
Michael Terry
c520fe8f19 PROD-1633: fix highlights error case
When calculating course highlights, if we didn't manage to get
a course module, we were previously throwing an exception.
Handle that more gracefully.
2020-06-09 11:05:02 -04:00
Adeel Khan
2383fb3fa6 Improving user locked out logic.
This patch improves on the user locked
out logic by providing a helping message
near locked out. This would help reduce
retries by giving user the option to use
password reset flow to fix the issue.

PROD-1505
2020-06-09 09:36:42 +05:00
Adeel Khan
ebc2948f81 Fix key error.
Job is failing because of unhandled
case where the mode, status key is
not found in course_cert_info.

PROD-1363
2020-06-08 15:53:03 +05:00
Michael Terry
2fbf9d1f29 Merge pull request #24154 from edx/mikix/courseware-api-marketing-url
Add marketing_url to courseware_api
2020-06-05 10:04:55 -04:00
Michael Terry
f88b4d75f0 Add marketing_url to courseware_api
This will let the courseware MFE point at a course's marketing page.

AA-137
2020-06-05 09:10:15 -04:00
Zainab Amir
cdc1c91980 Integrate Video Encode Manager (#24093)
* Add VEMPipelineIntegration config model
* Add course waffle flag to enable vem pipeline selectively

PROD-1636
2020-06-04 11:45:34 +05:00
Jeff LaJoie
7a45030de5 Merge pull request #24129 from edx/jlajoie/enable-highlights-job
Enables weekly highlights job for relative dates
2020-06-03 12:42:03 -04:00
Jeff LaJoie
5e6814c442 Enables weekly highlights job for relative dates 2020-06-03 12:07:56 -04:00