As in the previous post, I had created a simple web app using Python Flask to use as a teaching tool. The purpose was to demonstrate SQL injection and XSS (cross-site scripting) vulnerabilities and how to remediate them.<\/p>\n
In this case, the remediation step for XSS (escaping output) tripped me up. I tried this:<\/p>\n
return '<p>You searched for: ' + escape(user_input) + '<\/p>'<\/code><\/pre>\nI expected it to escape only the user_input<\/code> variable, but instead it escaped all the HTML, returning this:<\/p>\n<p>You searched for: <script>alert(1)<\/script><\/p><\/code><\/pre>\n
\n(Just want possible solutions? Scroll to the bottom. Otherwise, on to the….)<\/p>\n
Details<\/strong><\/p>\nThe reason for this is that Flask.escape()<\/a> returns a Markup<\/code> object, not a string.<\/p>\nBoth Markup<\/code> and escape<\/code> are imported into Flask from Jinja2<\/a>:<\/p>\nfrom jinja2 import escape\r\nfrom jinja2 import Markup<\/code><\/pre>\nWhich in turn comes from the module Markupsafe<\/a>. <\/p>\nMarkup<\/code> is a subclass of text_type<\/code> (which is essentially either str<\/code> or unicode<\/code>, depending on whether you are using Python2 or Python3).<\/p>\nThe Markup class contains __add__<\/code> and __radd__<\/code> methods that handle the behavior when we apply arithmetic operators (see Emulating numeric types<\/a>). In this case, those methods check to see if the other operand is compatible with strings, and if so, converts it to an escaped Markup<\/code> object as well:<\/p>\ndef __add__(self, other):\r\n if isinstance(other, string_types) or hasattr(other, \"__html__\"):\r\n return self.__class__(super(Markup, self).__add__(self.escape(other)))\r\n return NotImplemented\r\n\r\ndef __radd__(self, other):\r\n if hasattr(other, \"__html__\") or isinstance(other, string_types):\r\n return self.escape(other).__add__(self)\r\n return NotImplemented<\/code><\/pre>\n(From the source code at src\/markupsafe\/__init__.py<\/a>)<\/p>\nSurprising Results?<\/strong><\/p>\nAt first that seemed to me to violate the Principle Of Least Astonishment<\/a>. But I realized I didn’t know what Python would do if I created a subclass of str<\/code> and added a plain-old string to an object of my custom subclass. I decided to try it:<\/p>\n>>> import collections\r\n>>> class MyStringType(collections.UserString):\r\n... pass\r\n... \r\n>>> my_string1 = MyStringType(\"my test string\")\r\n>>> string1 = \"plain-old string\"\r\n>>> cat_string1 = my_string1 + string1\r\n>>> cat_string1\r\n'my test stringplain-old string'\r\n>>> type(my_string1)\r\n<class '__main__.MyString'>\r\n>>> type(string1)\r\n<class 'str'>\r\n>>> type(cat_string1)\r\n<class '__main__.MyString'><\/code><\/pre>\nInteresting, the result of adding an object of a subclass of string and a plain-old string is an object of the subclass! It turns out, the collections.UserString<\/code> object implements __add___<\/code> and __radd__<\/code> methods similar to what we saw in Markup<\/code>:<\/p>\ndef __add__(self, other):\r\n if isinstance(other, UserString):\r\n return self.__class__(self.data + other.data)\r\n elif isinstance(other, str):\r\n return self.__class__(self.data + other)\r\n return self.__class__(self.data + str(other))\r\ndef __radd__(self, other):\r\n if isinstance(other, str):\r\n return self.__class__(other + self.data)\r\n return self.__class__(str(other) + self.data)<\/code><\/pre>\n(From the source code at cpython\/Lib\/collections\/__init__.py<\/a>)<\/p>\nSolutions<\/strong><\/p>\nThere are several different ways to combine the escaped string (a Markup<\/code> object) and a string without converting the result to a Markup<\/code> object. The following is by no means an exhaustive list:<\/p>\nSolution 1: str.format<\/strong><\/p>\n>>> from flask import escape\r\n>>> user_input = '<script>alert(1);<\/script>'\r\n>>> html_output = '<p>You searched for: {}<\/p>'.format(escape(user_input))\r\n>>> html_output\r\n'<p>You searched for: <script>alert(1);<\/script><\/p>'<\/code><\/pre>\nSolution 2: printf-style string formatting<\/strong><\/p>\n>>> from flask import escape\r\n>>> user_input = '<script>alert(1);<\/script>'\r\n>>> html_output = '<p>You searched for: %s<\/p>' % escape(user_input)\r\n>>> html_output\r\n'<p>You searched for: <script>alert(1);<\/script><\/p>'<\/code><\/pre>\nSolution 3: cast the Markup<\/code> object to a str<\/code> object:<\/strong><\/p>\n>>> from flask import escape\r\n>>> user_input = '<script>alert(1);<\/script>'\r\n>>> html_output = '<p>You searched for: ' + str(escape(user_input)) + '<\/p>'\r\n>>> html_output\r\n'<p>You searched for: <script>alert(1);<\/script><\/p>'<\/code><\/pre>\nSolution 4: Create a Markup<\/code> object for the trusted HTML:<\/strong><\/p>\n>>> from flask import escape, Markup\r\n>>> user_input = '<script>alert(1);<\/script>'\r\n>>> html_output = Markup('<p>You searched for: ') + escape(user_input) + Markup('<\/p>')\r\n>>> html_output\r\nMarkup('<p>You searched for: <script>alert(1);<\/script><\/p>')<\/code><\/pre>\n(A Markup<\/code> object by default trusts the text passed to it on instantiation and does not escape it.)<\/p>\nA more likely solution, in practice, would be to use Jinja2 templates. Although Jinja2 templates do not automatically escape user input, they are configured to do so by Flask:<\/p>\n
Flask configures Jinja2 to automatically escape all values unless explicitly told otherwise. This should rule out all XSS problems caused in templates….<\/p><\/blockquote>\n
(from Security Considerations — Flask Documentation (1.1.x): Cross-Site Scripting<\/a>)<\/p>\n","protected":false},"excerpt":{"rendered":"As in the previous post, I had created a simple web app using Python Flask to use as a teaching tool. The purpose was to demonstrate SQL injection and XSS (cross-site scripting) vulnerabilities and how to remediate them. In this case, the remediation step for XSS (escaping output) tripped me up. I tried this: return … Continue reading Python Flask, escaping HTML strings, and the Markup class<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[86],"tags":[492,469,358,72],"class_list":["post-3137","post","type-post","status-publish","format-standard","hentry","category-python","tag-flask","tag-jinja2","tag-python","tag-xss"],"_links":{"self":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/3137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/comments?post=3137"}],"version-history":[{"count":7,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/3137\/revisions"}],"predecessor-version":[{"id":3144,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/3137\/revisions\/3144"}],"wp:attachment":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/media?parent=3137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/categories?post=3137"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/tags?post=3137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}