Instead of allowing the JSONDecodeError to escape, raise an error
indicating the request id and containing the first 100 characters
of the response to aid investigation.
For now, this only includes course id, search terms, and result count.
Information is logged both to the standard logger for near-term analysis
and the event tracker for the longer term. Result count logging requires
commit 02466b1 of cs_comments_service (otherwise it will be null).
Previously, an error was raised if the comments service returned data
including an unexpected field, which unnecessarily complicated the
release path for new features, since the list of allowed fields would
need to be modified before cs_comments_service could be modified, and
only then could edx-platform take advantage of the new CS feature. We
still log a warning if an unexpected field is returned, so we will
still be able to tell if the CS returns a corrupt response.
JIRA: FOR-180
This commit adds the non-courseware lms/djangoapps and lms/lib.
These keys are now objects with a limited interface, and the particular
internal representation is managed by the data storage layer (the
modulestore).
For the LMS, there should be no outward-facing changes to the system.
The keys are, for now, a change to internal representation only. For
Studio, the new serialized form of the keys is used in urls, to allow
for further migration in the future.
Co-Author: Andy Armstrong <andya@edx.org>
Co-Author: Christina Roberts <christina@edx.org>
Co-Author: David Baumgold <db@edx.org>
Co-Author: Diana Huang <dkh@edx.org>
Co-Author: Don Mitchell <dmitchell@edx.org>
Co-Author: Julia Hansbrough <julia@edx.org>
Co-Author: Nimisha Asthagiri <nasthagiri@edx.org>
Co-Author: Sarina Canelake <sarina@edx.org>
[LMS-2370]
This captures real-time metrics for all of the comment client actions,
segregated by course_id, as well as other small-cardinality fields.
The goal is to be able to detect changes in forum usage, with the goal
of alerting on potential error conditions.
Also improves handling in cases where account creation did succeed, but
comments service user creation did not.
Depends on commit 88abdd8a0b20bc9816f487b25abdea1953f5661d in https://github.com/edx/cs_comments_service
JIRA: FOR-522
This change requires cs_comments_service version 31ef160 or later. Now
that the /threads endpoint can filter by commentable_ids, use that
instead of the /search/threads endpoint, which does not sort and
paginate correctly.
Bug: FOR-224
CommentClientError now has sane subclasses that are meaningfully
distinct, and each subclass is handled appropriately. Errors raised by
the requests library are no longer handled by turning them into
CommentClientErrors, since there is no meaningful handling we can do,
and this way we will get more visibility into why errors are occurring.
Also, HTTP status codes from the comments service indicating client
error are correctly passed through to the client.
Logging the duration of each request will allow us to determine
whether there is a significant difference in the latency reported by
the comments service and that observed by the LMS. Each request will
be assigned a unique identifier to allow correlation of the reported
latency on each end.
Previously, authentication was done using a URL parameter, which would
appear in various logs. Now, authentication is done more appropriately
with an HTTP header. Note that this requires cs_comments_service commit
cf39aabdd160176ebf206ca19d3ee030161a0b47 or later.
According to someone from Datadog, this was generating tags like "knowledgeable_
people_who_put_this_course_together._this_is_harvard._you_can_t_tell_us_there_s_
a_shortage_of_editorial_talent." They say that they can handle tens or hundreds
of unique tags but not thousands. Given that we have a unique URL for each
thread, we can't even use that as a tag. Thus, all tags are removed for now
until we can determine whether there is a useful set of tags with small enough
cardinality. In light of this, I did not investigate why the long tag mentioned
above was being generated.
The LMS comment client previously would try to parse the response
as JSON, choke, and return a 500 to the client. Now, the LMS client
displays a message indicating that the forums are down for
maintenance.
