[Laszlo-dev] [Updated] For Review: Change 20090608-raju-Y Summary: Fix for CSS parser uses incorrect file encoding
Raju Bitter
rajubitter at me.com
Wed Jun 10 07:10:43 PDT 2009
Tucker already did that. So it's in 4.4:
Atlantia:4.4 rajubitter$ svn log
------------------------------------------------------------------------
r14096 | ptw | 2009-06-09 19:47:22 +0200 (Tue, 09 Jun 2009) | 84 lines
Merged revisions 14091-14095 via svnmerge from
http://svn.openlaszlo.org/openlaszlo/trunk
........
r14091 | raju | 2009-06-09 12:25:06 -0400 (Tue, 09 Jun 2009) | 52
lines
Change 20090608-raju-Y by raju at ip-90-186-160-71.web.vodafone.de on
2009-06-08 20:09:02 CEST
in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
for http://rajubitter@svn.openlaszlo.org/openlaszlo/trunk
On Jun 10, 2009, at 2:15 PM, Amy Muntz wrote:
> Thanks, Raju.
> -- Amy
>
> On Tue, Jun 9, 2009 at 6:56 PM, Raju Bitter<rajubitter at me.com> wrote:
>> Sorry, Amy, just saw that mail now. I'll do that tomorrow.
>>
>> Best,
>> Raju
>>
>> On Jun 9, 2009, at 4:41 PM, Amy Muntz wrote:
>>
>>> Hi Raju,
>>> Please check in the trunk fix for 4.4 right away. It may take an
>>> additional fix for 4.0.x, but that's just fine.
>>> We can re-open the bug for 4.0.x and it can be fixed at a later
>>> time.
>>> Thanks,
>>> Amy
>>>
>>>
>>> On Mon, Jun 8, 2009 at 7:08 PM , Raju Bitter wrote:
>>>
>>>> Tucker, Wanli,
>>>>
>>>> I've done some testing with OL 4.0.15 and Webtop running 1.5.2
>>>> running on
>>>> OL 4.0.15. While the fix works for OL trunk, it doesn't work for
>>>> 4.0.15.
>>>> Here's the debug output for test file
>>>> "test/css/encoding/utf8_with_BOM_no_charset_attr.lzx". The UTF-8
>>>> encoded
>>>> data is read correctly.
>>>>
>>>>
>>>> http://localhost:8080/4.0.15/test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>>>>>
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>>>>> compiler.StyleSheetCompiler -
>>>>> StyleSheetCompiler.compile called!
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>>>>> compiler.StyleSheetCompiler -
>>>>> @charset=utf-8 found on stylesheet tag
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO
>>>>> compiler.StyleSheetCompiler -
>>>>> reading in stylesheet from src="utf8_with_BOM_no_charset_attr.css"
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
>>>>> Resolving pathname: utf8_with_BOM_no_charset_attr.css and base: /
>>>>> Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/encoding
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG compiler.FileResolver -
>>>>> Resolved utf8_with_BOM_no_charset_attr.css to /Users/rajubitter/
>>>>> src/
>>>>> svn/openlaszlo/4.0.15/test/css/encoding/
>>>>> utf8_with_BOM_no_charset_attr.css
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG cm.DependencyTracker -
>>>>> addFile
>>>>> Path is /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/css/
>>>>> encoding/utf8_with_BOM_no_charset_attr.css
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
>>>>> creating CSSHandler
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) INFO css.CSSHandler -
>>>>> Trying
>>>>> to parse CSS with charset setting of utf-8
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>>>>> Read
>>>>> the following bytes: text[name='german'] {
>>>>> backgroundColor: #ffcc00;
>>>>> buttonText: "öäüÖÄܧ";
>>>>> }
>>>>>
>>>>> text[nam
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>>>>> Testing
>>>>> for UTF-8 BOM!
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG utils.FileUtils -
>>>>> Found
>>>>> BOM on file, encoding is UTF-8
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
>>>>> Opening
>>>>> CSS file /Users/rajubitter/src/svn/openlaszlo/4.0.15/test/
>>>>> css/encoding/utf8_with_BOM_no_charset_attr.css using encoding
>>>>> UTF-8
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG css.CSSHandler -
>>>>> Skip
>>>>> first 3 bytes containing UTF-8 BOM
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> compiling CSSHandler using new unique names
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Conditional selector: [name="german"]
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Attribute condition
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> simple selector:text
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Cond string: { attrname: "name", attrvalue: "german",
>>>>> simpleselector:
>>>>> "text"}
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Conditional selector: [name="korean"]
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Attribute condition
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> simple selector:text
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Cond string: { attrname: "name", attrvalue: "korean",
>>>>> simpleselector:
>>>>> "text"}
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Conditional selector: [name="chinese"]
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Attribute condition
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> simple selector:text
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> Cond string: { attrname: "name", attrvalue: "chinese",
>>>>> simpleselector:
>>>>> "text"}
>>>>> 09 Jun 2009 01:03:06 (127.0.0.1 15) DEBUG
>>>>> compiler.StyleSheetCompiler -
>>>>> whole stylesheet as css $lzc $style._addRule(new $lzc
>>>>> $rule({ attrname:
>>>>> "name", attrvalue: "german", simpleselector: "text"},
>>>>> {backgroundColor:
>>>>> 0xFFCC00, buttonText: "???????"}));
>>>>> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>>>>> "korean", simpleselector: "text"}, {backgroundColor: 0xFF00CC,
>>>>> buttonText:
>>>>> "? ?? ??? ?? ?? ?? ?? ??"}));
>>>>> $lzc$style._addRule(new $lzc$rule({ attrname: "name", attrvalue:
>>>>> "chinese", simpleselector: "text"}, {backgroundColor: 0x00FFCC,
>>>>> buttonText:
>>>>> "???????????????"}))
>>>>
>>>> But the debug console in the browser shows the following error
>>>> message:
>>>>>
>>>>> ERROR: Unknown cohort for rule: #0 * (undefined)
>>>>> ERROR: Unknown cohort for rule: #1 * (undefined)
>>>>> ERROR: Unknown cohort for rule: #2 * (undefined)
>>>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found
>>>>> for
>>>>> node «lz.text#3| #german» for property name buttonText
>>>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#7: No CSS value found
>>>>> for
>>>>> node «lz.text#3| #german» for property name backgroundColor
>>>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found
>>>>> for
>>>>> node «lz.text#4| #korean» for property name buttonText
>>>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#8: No CSS value found
>>>>> for
>>>>> node «lz.text#4| #korean» for property name backgroundColor
>>>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found
>>>>> for
>>>>> node «lz.text#5| #chinese» for property name buttonText
>>>>> WARNING @utf8_with_BOM_no_charset_attr.lzx#9: No CSS value found
>>>>> for
>>>>> node «lz.text#5| #chinese» for property name backgroundColor
>>>>
>>>> Any idea what this might be?
>>>>
>>>> - Raju
>>>>
>>>>
>>>> On Jun 8, 2009, at 9:42 PM, P T Withington wrote:
>>>>
>>>>> [cc-ing Wanli]
>>>>>
>>>>> Wanli,
>>>>>
>>>>> Maybe you also want to review (or have someone on your team
>>>>> review) as
>>>>> this issue affects you?
>>>>>
>>>>> ---
>>>>>
>>>>> Raju,
>>>>>
>>>>> This looks good.
>>>>>
>>>>> I think you might have trouble trying checking in your utf-16
>>>>> examples
>>>>> to svn. I'm not sure svn will do the right thing. It looks
>>>>> like it is
>>>>> treating them as binary files. Maybe that will work. I don't
>>>>> know.
>>>>>
>>>>> It looks to me like you may also have some tab characters in your
>>>>> sources. You need to expand tabs to check in, otherwise the
>>>>> pre- commit
>>>>> filter will reject your checkin.
>>>>>
>>>>> I approve this change. You might want to check in your fix
>>>>> separately
>>>>> from your tests, just in case the UTF-16 files cause svn issues.
>>>>>
>>>>> On 2009-06-08, at 14:44EDT, Raju Bitter wrote:
>>>>>
>>>>>> Added a "+" in debug output string concatenation.
>>>>>>
>>>>>> Change 20090608-raju-Y by raju at ip-90-186-160-71.web.vodafone.de
>>>>>> on
>>>>>> 2009-06-08 20:09:02 CEST
>>>>>> in /Users/rajubitter/src/svn/openlaszlo/trunk-cssunicode
>>>>>> for http://rajubitter@svn.openlaszlo.org/openlaszlo/trunk
>>>>>>
>>>>>> Summary: Fix for CSS parser uses incorrect file encoding
>>>>>>
>>>>>> New Features: Adds an optional @charset to the stylesheet tag,
>>>>>> in case
>>>>>> the user wants to use a CSS file in a different encoding then
>>>>>> utf-8
>>>>>>
>>>>>> Bugs Fixed: LPP-8045
>>>>>>
>>>>>> Technical Reviewer: ptw
>>>>>> QA Reviewer: (pending)
>>>>>> Doc Reviewer: (pending)
>>>>>>
>>>>>> Documentation:
>>>>>>
>>>>>> Release Notes:
>>>>>>
>>>>>> Details:
>>>>>> + StyleSheetCompiler.java: Handling for @charset on stylesheet
>>>>>> tag
>>>>>> added. Added a
>>>>>> 2nd parameter with the encoding value to the CSSHandler.parse()
>>>>>> call.
>>>>>>
>>>>>> + CSSHandler.java:
>>>>>> parse() method takes the encoding from the LZX stylesheet tag
>>>>>> as a 2nd
>>>>>> parameter.
>>>>>> getInputSource() method does a few more things now:
>>>>>> 1) checks for a possible BOM on the CSS file
>>>>>> 2) checks if a possible BOM conflicts with the value of the
>>>>>> stylesheet
>>>>>> tag's @charset
>>>>>> 3) if there's a BOM, the BOM bytes are removed from the input
>>>>>> stream
>>>>>>
>>>>>> + FileUtils.java:
>>>>>> Added method public static String
>>>>>> detectBOMEncoding(BufferedInputStream
>>>>>> in)
>>>>>> The method returns the BOM marker interpreted as one of the
>>>>>> following
>>>>>> strings:
>>>>>> UTF-8
>>>>>> UTF-16LE
>>>>>> UTF-16BE
>>>>>>
>>>>>> Tests:
>>>>>> + test files in folder test/css/encoding
>>>>>> The following test exist:
>>>>>> 1) iso8859-1_with_charset_attr.lzx
>>>>>> Reading an iso-8859-2 encoded CSS file with some German
>>>>>> special chars
>>>>>> 2) utf16BE_with_BOM.lzx
>>>>>> Reading an utf-16 BE CSS file with BOM marker
>>>>>> 3) utf16LE_with_BOM.lzx
>>>>>> Reading an utf-16 LE CSS file with BOM marker
>>>>>> 4) utf8_with_BOM_no_charset_attr.lzx
>>>>>> Reading an utf-8 CSS file with BOM and no charset attribute on
>>>>>> the
>>>>>> stylesheet tag
>>>>>> 5) utf8_with_BOM_conflicting_charset_attr.lzx
>>>>>> Reading a CSS with @charset value of utf-16, but CSS having a
>>>>>> UTF-8
>>>>>> BOM marker, will
>>>>>> throw a compile error
>>>>>>
>>>>>> Files:
>>>>>> A test/css/encoding
>>>>>> A test/css/encoding/iso8859-1_with_charset_attr.lzx
>>>>>> A test/css/encoding/utf16BE_with_BOM.lzx
>>>>>> A test/css/encoding/utf8_with_BOM_no_charset_attr.css
>>>>>> A test/css/encoding/utf16LE_with_BOM.css
>>>>>> A test/css/encoding/utf8_no_BOM_no_charset_attr.css
>>>>>> A test/css/encoding/
>>>>>> utf8_with_BOM_conflicting_charset_attr.css
>>>>>> A test/css/encoding/utf8_with_BOM_no_charset_attr.lzx
>>>>>> A test/css/encoding/utf16LE_with_BOM.lzx
>>>>>> A test/css/encoding/utf8_no_BOM_no_charset_attr.lzx
>>>>>> A test/css/encoding/iso8859-1_with_charset_attr.css
>>>>>> A test/css/encoding/utf16BE_with_BOM.css
>>>>>> A test/css/encoding/
>>>>>> utf8_with_BOM_conflicting_charset_attr.lzx
>>>>>> M WEB-INF/lps/server/src/org/openlaszlo/css/CSSHandler.java
>>>>>> M WEB-INF/lps/server/src/org/openlaszlo/utils/
>>>>>> FileUtils.java
>>>>>> M WEB-INF/lps/server/src/org/openlaszlo/compiler/
>>>>>> StyleSheetCompiler.java
>>>>>>
>>>>>> Changeset:
>>>>>> http://svn.openlaszlo.org/openlaszlo/patches/20090608-raju-Y.tar
>>>>>
>>
>>
>>
More information about the Laszlo-dev
mailing list