cb58873879
The cache in the previous version of this code was unwittingly being shared among all threads, and an occasional race condition would result in the .children field of some XBlocks containing duplicate entries. I tried to find other ways to keep the existing cache design and let it be shared among all the threads (which would be more efficient), but I couldn't find any clean way of doing it (and even then, this code was not written with the intention of being used in a multi-threaded way). So to keep the fix simple, I made the block data cache thread-local instead of process-local. That eliminated the bug. Technical details: The big challenge with this code in the first place was due to the parse_xml API of XBlocks, which instantiates the XBlock instance and _then_ sets field data on it and returns it, as there is no mechanism available to distinguish that from the case of instantiating an XBlock (using previously loaded/cached field data) and then deliberately changing its field values. In particular, parse_xml sets the 'children' field just by calling self.children.append(...) but never explicitly initializes self.children to [] first, so it's necessary for the field data store to have a mechanism for setting self.children = [] when parsing begins, but it's hard to know "when parsing begins" in general. This is made more challenging since the BlockstoreFieldData design ties field data changes to the XBlock instance itself (using a weakkey), but it's impossible to get a reference to the XBlock instance before calling parse_xml, and since the BlockstoreFieldData design uses a pass-through approach where fields that aren't being actively changed are read from the cache; since it doesn't know when children is being initialized vs. being modified, it would sometimes pass-through and start one thread's changes with the final result from another thread. Anyhow, the bottom line is that avoiding unintentional multithreading here solves the problem. If we want the field data cache to be shared among threads, it might as well be rewritten to use memcached and shared among all processes too. That would be a very good performance boost but would take up a lot more memory in memcached. Also the rewrite may be challenging due to the aforementioned nuances of the XBlock parse_xml / construct_xblock / add_node_as_child APIs. Perhaps modifying the runtime to use a completely separate fielddata implementation for parsing vs. loading a parsed+cached XBlock would do it.
This is the core repository of the Open edX software. It includes the LMS
(student-facing, delivering courseware), and Studio (course authoring)
components.
Installation
------------
Installing and running an Open edX instance is not simple. We strongly
recommend that you use a service provider to run the software for you. They
have free trials that make it easy to get started:
https://openedx.org/get-started/
If you will be modifying edx-platform code, the `Open edX Developer Stack`_ is
a Docker-based development environment.
If you want to run your own Open edX server and have the technical skills to do
so, `Open edX Ubuntu 16.04 64-bit Installation`_ has instructions to install
it on an existing Ubuntu 16.04 server.
.. _Open edX Developer Stack: https://github.com/edx/devstack
.. _Open edX Ubuntu 16.04 64-bit Installation: https://openedx.atlassian.net/wiki/display/OpenOPS/Native+Open+edX+Ubuntu+16.04+64+bit+Installation
License
-------
The code in this repository is licensed under version 3 of the AGPL
unless otherwise noted. Please see the `LICENSE`_ file for details.
.. _LICENSE: https://github.com/edx/edx-platform/blob/master/LICENSE
More about Open edX
-------------------
See the `Open edX site`_ to learn more about the Open edX world. You can find
information about hosting, extending, and contributing to Open edX software. In
addition, the Open edX site provides product announcements, the Open edX blog,
and other rich community resources.
.. _Open edX site: https://openedx.org
Documentation
-------------
Documentation can be found at https://docs.edx.org.
Getting Help
------------
If you're having trouble, we have discussion forums at
https://discuss.openedx.org where you can connect with others in the community.
Our real-time conversations are on Slack. You can request a `Slack
invitation`_, then join our `community Slack team`_.
For more information about these options, see the `Getting Help`_ page.
.. _Slack invitation: https://openedx-slack-invite.herokuapp.com/
.. _community Slack team: http://openedx.slack.com/
.. _Getting Help: https://openedx.org/getting-help
Issue Tracker
-------------
We use JIRA for our issue tracker, not GitHub issues. You can search
`previously reported issues`_. If you need to report a problem,
please make a free account on our JIRA and `create a new issue`_.
.. _previously reported issues: https://openedx.atlassian.net/projects/CRI/issues
.. _create a new issue: https://openedx.atlassian.net/secure/CreateIssue.jspa?issuetype=1&pid=11900
How to Contribute
-----------------
Contributions are welcome! The first step is to submit a signed
`individual contributor agreement`_. See our `CONTRIBUTING`_ file for more
information – it also contains guidelines for how to maintain high code
quality, which will make your contribution more likely to be accepted.
Reporting Security Issues
-------------------------
Please do not report security issues in public. Please email
security@edx.org.
.. _individual contributor agreement: https://openedx.org/wp-content/uploads/2019/01/individual-contributor-agreement.pdf
.. _CONTRIBUTING: https://github.com/edx/edx-platform/blob/master/CONTRIBUTING.rst