[Laszlo-dev] [Updated] For Review: Change 20090608-raju-Y Summary: Fix for CSS parser uses incorrect file encoding

[Raju Bitter]

Added a "+" in debug output string concatenation.

Change 20090608-raju-Y by raju at ip-90-186-160-71.web.vodafone.de on  
2009-06-08 20:09:02 CEST
     in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
     for http://rajubitter@svn.openlaszlo.org/openlaszlo/trunk

Summary: Fix for CSS parser uses incorrect file encoding

New Features: Adds an optional @charset to the stylesheet tag, in case  
the user wants to use a CSS file in a different encoding then utf-8

Bugs Fixed: LPP-8045

Technical Reviewer: ptw
QA Reviewer: (pending)
Doc Reviewer: (pending)

Documentation:

Release Notes:

Details:
+ StyleSheetCompiler.java: Handling for @charset on stylesheet tag  
added. Added a
2nd parameter with the encoding value to the CSSHandler.parse() call.

+ CSSHandler.java:
parse() method takes the encoding from the LZX stylesheet tag as a 2nd  
parameter.
getInputSource() method does a few more things now:
   1) checks for a possible BOM on the CSS file
   2) checks if a possible BOM conflicts with the value of the  
stylesheet tag's @charset
   3) if there's a BOM, the BOM bytes are removed from the input stream

+ FileUtils.java:
Added method public static String  
detectBOMEncoding(BufferedInputStream in)
The method returns the BOM marker interpreted as one of the following  
strings:
   UTF-8
   UTF-16LE
   UTF-16BE

Tests:
+ test files in folder test/css/encoding
The following test exist:
   1) iso8859-1_with_charset_attr.lzx
      Reading an iso-8859-2 encoded CSS file with some German special  
chars
   2) utf16BE_with_BOM.lzx
      Reading an utf-16 BE CSS file with BOM marker
   3) utf16LE_with_BOM.lzx
      Reading an utf-16 LE CSS file with BOM marker
   4) utf8_with_BOM_no_charset_attr.lzx
      Reading an utf-8 CSS file with BOM and no charset attribute on  
the stylesheet tag
   5) utf8_with_BOM_conflicting_charset_attr.lzx
      Reading a CSS with @charset value of utf-16, but CSS having a  
UTF-8 BOM marker, will
      throw a compile error

Files:
A       test/css/encoding
A       test/css/encoding/iso8859-1_with_charset_attr.lzx
A       test/css/encoding/utf16BE_with_BOM.lzx
A       test/css/encoding/utf8_with_BOM_no_charset_attr.css
A       test/css/encoding/utf16LE_with_BOM.css
A       test/css/encoding/utf8_no_BOM_no_charset_attr.css
A       test/css/encoding/utf8_with_BOM_conflicting_charset_attr.css
A       test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
A       test/css/encoding/utf16LE_with_BOM.lzx
A       test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
A       test/css/encoding/iso8859-1_with_charset_attr.css
A       test/css/encoding/utf16BE_with_BOM.css
A       test/css/encoding/utf8_with_BOM_conflicting_charset_attr.lzx
M       WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
M       WEB-INF/lps/server/src/org/openlaszlo/utils/FileUtils.java
M       WEB-INF/lps/server/src/org/openlaszlo/compiler/ 
StyleSheetCompiler.java

Changeset: http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar