updated the csv reader to make the csv output compatible across both python versions
added the encoding argument to unicodecsv.DictReader method
updated the csv file read process to accomodate different input types
changes made to fix issues with UniversalNewLineIterator
changes made as suggested
changes made as suggested
changes made to fulfill quality check requirement to pass quality check
minor changes
So making this code change took a few hours. But then deciding that it
was the right one of the many options available took the next 3 days.
When changing to the new rounding function, we had a test that started
failing. It appears as if our new rounding function is not the same in
some way as the one built into python 2.
```
>>> round(.0045*100 + .05)/100
0.0
>>> round_away_from_zero(.0045*100 + .05)/100
0.01
```
Doing the math by hand we see that the new function is actually correct
but the old one is clearly rounding incorrectly in this case. Looking
closer at this I discovered that it was due to a floating point issue
where .0045*100 is represented as 0.44999999999999996 so when we add
0.05 to this number we get 0.49999999999999994. This is all because of
the limitations of floating point arithmetic.
See https://docs.python.org/3/tutorial/floatingpoint.html#tut-fp-issues
for more on that.
Because python does its rounding at the bit level in C code. It treats
the .4999... as below the .5 cutoff and rounds down. Whereas our code
does more simple arithmetic which causes the number to correct itself
before we round and so correctly rounds up to 0.01
The result of this change is that previously, the rounding threshold used to
be that any number > .0045 would ronud to 0.01 and now any number that
is >= .0045 rounds to 0.01
Note that if we only care about the two most significant digits of
number between 0 and 1, this error only manifests itself in such a way
that other than the case of going from 0.00 to 0.01 eg. from 0% to 1%
none of the other cases where we would now round up cause the 2 most
significant digits to change. Given this level of impact, we're fine
with this change.
In our tests we see this for one case, where an incomplete turns into an
F in a test. I'm updating the test here to be more correct.
As we were looking at it we speculated as to why we were adding the .05
to the number. Could it be to counteract this floating point issue? It
turns out not.
Looking at this commit(a1286b1c7d) we see that it
looks like this was intended to always round up to the nearest
percentage point. However, there's a typo here. If you wanted to
ensure that we always rounded up to the nearest percentage point you
would have the math be `round(final_grade_percent*100 + 0.5)/ 100` or a
simpler way to do this would be
`math.ceil(final_grade_percent*100)/100`. However, that is not what
happened and 7 years later, there have been a lot of people graded with
the wrong rounding where essentialy anyone with a grade of 89.45 gets a
90 when the intended impact was supposed to be that anyone with a grade
above an 89.0 would get a grade of 90.
Changing it now requires a lot of conversation and wolud have a large
impact on existing learners. So we are not going to change it as a part
of the python 2 -> python 3 upgrade. I have created
https://openedx.atlassian.net/browse/TNL-6972 to capture this issue if
we want to address it in the future.
See the commit that added round_away_from_zero for details.
IN the case of dashboard_data.py:
We don't first cast to decimal here because in python 2 this didn't do
anything. The python 2 rounding method would just cast things to float
before doing the rounding anyway so it just wastes cpu cycles. In
python 3 this causes some typing issues so just removing it.
ecommerce_worker is the only service user with a @fake.email instead of
an @example.com email address. This was fixed manually in a devstack sql
dump, but was lost during the next sql dump. This will fix it moving
forward so we don't have to work around it.
* Fix type mismatches in coursewaqre
* Fix type mismatch in credit migrations
* Fix type mismatch in status migrations
* Fix type mismatch in user_api migrations
* Review Fixes
Learners are not allowed to make an attempt of the procotored exam if
they verify their identity near to proctored exam date.To make them,
aware about their expiry date, modification are done to the status card
so that user experience will be improved.
PROD-769
changes made to finalize the url sanitization to make url comparison consistent
changes made to avoid running sanitize in python 2
changes made to substitute proper query param in url for comparison
changes made to split and compare the query parameter with expected url
updated the split logic to properly split/slice url parts for comparison
updated the dashboard template to remove encode and avoid bytes encoded embeded url
updated the dashboard template to enclose social media urls with unicode
* fix type mismatch in third_party_auth migrations
* fix type mismatch in verify_student migrations
* fix type mismatch in video_config migrations
* fix type mismatch in verified_track_content migrations
* fix type mismatch in commercemigrations
* fix type mismatch in xblock_config migrations
* fix type mismatch in course_creators migrations
* fix type mismatch in contentstore migrations
* fix type mismatch in course_goals migrations
* fix type mismatch in experiments migrations
* fix type mismatch in xblock_django migrations
* fix type mismatch in catalog migrations
* fix type mismatch in static_replace migrations
* fix type mismatch in bulk_email migrations
* fix type mismatch in course_overviews migrations
* Fix type mismatches in the course_modes migrations
* Fix type mismatches in the course_action_state migrations
* Fix type mismatches in the schedules migrations
* Fix type mismatches in grades migrations
* Fix type mismatches in video_pipeline
* Fix type mismatches in api_admin
* Fix mismatches in credential migrations
