By default, Bugzilla does not search the list of RESOLVED bugs.
You can force it to do so by putting the upper-case word ALL in front of your search query, e.g.: ALL tdelibs
We recommend searching for bugs this way, as you may discover that your bug has already been resolved and fixed in a later release.
Bug 2613 - VCardTool::parseDateTime is broken
Summary: VCardTool::parseDateTime is broken
Status: RESOLVED FIXED
Alias: None
Product: TDE
Classification: Unclassified
Component: tdelibs (show other bugs)
Version: R14.0.1 [Trinity]
Hardware: amd64 Linux
: P5 normal
Assignee: Michele Calgaro
URL:
Depends on:
Blocks: R14.0.4
  Show dependency treegraph
 
Reported: 2016-03-16 20:01 CDT by deloptes
Modified: 2016-04-03 06:26 CDT (History)
4 users (show)

See Also:
Compiler Version:
TDE Version String:
Application Version:
Application Name:


Attachments
proposal to fix parseDateTime (1.19 KB, text/x-c++src)
2016-03-16 20:04 CDT, deloptes
Details
proposal to fix parseDateTime v2 (477 bytes, text/x-c++src)
2016-03-18 04:57 CDT, deloptes
Details
Patch for VCardTool::parseDateTime() (5.15 KB, patch)
2016-03-22 20:37 CDT, Michele Calgaro
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description deloptes 2016-03-16 20:01:44 CDT
Hi,
I'm sorry to cite my html doc, but I do not have the original source code.

I need a quick fix for this issue. 

Revision is in ISO 8601 format - ex: https://en.wikipedia.org/wiki/ISO_8601

But according this both 2016-03-16T15:26:53Z and 20160316T152653Z are valid.

However if I feed parseVCard with 20160316T152653Z I get

TQDate::setYMD: Invalid date 0000-00-00
TQTime::setHMS Invalid time 26:00:00.000

If I feed it with 2016-03-16T15:26:53Z I get

TQDate::setYMD: Invalid date 0000-00-00

Looking into the issue in VCardTool::parseDateTime originally you have

609 TQDateTime VCardTool::parseDateTime( const TQString &str )
610 {
611  TQDateTime dateTime;
612 
613  if ( str.find( '-' ) == -1 ) { // is base format (yyyymmdd)
614  dateTime.setDate( TQDate( str.left( 4 ).toInt(), str.mid( 4, 2 ).toInt(),
615  str.mid( 6, 2 ).toInt() ) );
616 
617  if ( str.find( 'T' ) ) // has time information yyyymmddThh:mm:ss
618  dateTime.setTime( TQTime( str.mid( 11, 2 ).toInt(), str.mid( 14, 2 ).toInt(),
619  str.mid( 17, 2 ).toInt() ) );
620 
621  } else { // is extended format yyyy-mm-dd
622  dateTime.setDate( TQDate( str.left( 4 ).toInt(), str.mid( 5, 2 ).toInt(),
623  str.mid( 8, 2 ).toInt() ) );
624 
625  if ( str.find( 'T' ) ) // has time information yyyy-mm-ddThh:mm:ss
626  dateTime.setTime( TQTime( str.mid( 11, 2 ).toInt(), str.mid( 14, 2 ).toInt(),
627  str.mid( 17, 2 ).toInt() ) );
628  }
629 
630  return dateTime;
631 }

Which is not correct.

This needs to change to

 TQDateTime VCardTool::parseDateTime( const TQString &str )
 {
  TQDateTime dateTime;
 
  if ( str.find( '-' ) == -1 ) { // is base format (yyyymmdd)
  	dateTime.setDate( TQDate( str.left( 4 ).toInt(), str.mid( 4, 2 ).toInt(),
  		str.mid( 6, 2 ).toInt() ) );
 
  	if ( str.find( 'T' ) ) { // has time information yyyymmddThh:mm:ss
	    if ( str.find( ':' ) )
  		dateTime.setTime( TQTime( str.mid( 9, 2 ).toInt(), str.mid( 12, 2 ).toInt(),
  			str.mid( 15, 2 ).toInt() ) ); 
	    else // has time information yyyymmddThhmmss
  		dateTime.setTime( TQTime( str.mid( 9, 2 ).toInt(), str.mid( 11, 2 ).toInt(),
  			str.mid( 13, 2 ).toInt() ) );
	}
 
  } else { // is extended format yyyy-mm-dd
  	dateTime.setDate( TQDate( str.left( 4 ).toInt(), str.mid( 5, 2 ).toInt(),
  		str.mid( 8, 2 ).toInt() ) );
 
  	if ( str.find( 'T' ) ) { // has time information yyyy-mm-ddThh:mm:ss
	    if ( str.find( ':' ) )
  		dateTime.setTime( TQTime( str.mid( 11, 2 ).toInt(), str.mid( 14, 2 ).toInt(),
  			str.mid( 17, 2 ).toInt() ) ); 
	    else // has time information yyyy-mm-ddThhmmss
  		dateTime.setTime( TQTime( str.mid( 11, 2 ).toInt(), str.mid( 13, 2 ).toInt(),
  			str.mid( 15, 2 ).toInt() ) );
	}
  }
 
  return dateTime;
 }

This still does not cover ISO 8601 fully because we miss the Z+ information, however for PIM I think it should be sufficient
Comment 1 deloptes 2016-03-16 20:04:58 CDT
Created attachment 2637 [details]
proposal to fix parseDateTime
Comment 2 Michele Calgaro 2016-03-17 02:33:57 CDT
Let's fix it properly, including support for the 'Z' format.
Added to R14.0.4 bug list. I will work on this during the weekend or at latest next week.
I changed back priority to normal, since it is not a blocker for the whole TDE project.
Comment 3 deloptes 2016-03-17 04:20:29 CDT
One note on the Z format.

20160316T152653Z is valid, so a check behind Z resulting in empty should be valid. The part as I understand is +/- signed time offset

thanks in advance
Comment 4 deloptes 2016-03-18 04:57:34 CDT
Created attachment 2638 [details]
proposal to fix parseDateTime v2

After thinking for a while in the past few days, it looks to me that the solution in the function is over-complicated.
I suggest a new solution.
I'll test it next and report back.
Comment 5 Michele Calgaro 2016-03-18 07:09:26 CDT
Actually I am thinking to use regex to make sure the match is exact.
But if you have other patches fill free to attach them, I will check them all.
Cheers
  Michele
Comment 6 deloptes 2016-03-18 16:31:47 CDT
Hi Michele,
the v2 I proposed works fine. I'll attach the patch in couple of minutes.

I do not encourage you to use regex when unnecessary and in this case the approach with removing '-' and ':' is elegant and confirms to ISO standard. 
You have to match 'T' and if you want to add awareness of time zone - match 'Z'.
The ISO is pretty clear what is allowed and what not.

I did not answer that fast, because I was still seeing
TQDate::setYMD: Invalid date 0000-00-00. 

It took me a while (actually I had to write a test program) to understand what was wrong.

It turns out if you feed parseVCard with addresse, where birthday is empty "BDAY:\n", it would feed empty string to parseDateTime, which in my opinion is not correct.

We may need to rework this part as well via another ticket. I am not quite sure how this influences my work here atm.

regards
Comment 7 deloptes 2016-03-18 18:31:25 CDT
Looking again into the details of the ISO 8601 spec, I understand much better why they have taken this approach for parsing ... and never finished/implemented it right. 

Perhaps regex is more appropriate - I don't know. If you are not using UTC (Z) and pass some TZ offset it will not work with the old and new code, but there is a TZ key in the vCard format - I'm a bit confused. One should check the vCard specs, but I did not do it so far.

So what means doing this in appropriate way? add an ISO8601 parser/converter? The version we have also does not understand 'W' etc.

If you want the patch, let me know. 

For BDAY, I suggest this (it works for me)

--- orig/tdelibs-trinity-14.0.3/tdeabc/vcardtool.cpp    2013-09-03 21:00:38.000000000 +0200
+++ tdelibs-trinity-14.0.3/tdeabc/vcardtool.cpp 2016-03-19 00:20:47.694643684 +0100
@@ -412,9 +412,11 @@
           addr.setAgent( parseAgent( *lineIt ) );

         // BDAY
-        else if ( identifier == "bday" )
-          addr.setBirthday( parseDateTime( (*lineIt).value().asString() ) );
-
+        else if ( identifier == "bday" ) {
+         TQString s((*lineIt).value().asString());
+         if ( s.length() > 0 )
+             addr.setBirthday( parseDateTime( s ) );
+       }


regards
Comment 8 Michele Calgaro 2016-03-19 04:21:19 CDT
IMO regexs are perfect for this task, since we have to verify that the string adhears to a specific format.

As for what implement, I would ssay to take example from here
https://www.w3.org/TR/NOTE-datetime
https://en.wikipedia.org/wiki/ISO_8601
If we support all those formats, we should be ok :-)

What do you think?
Comment 9 Michele Calgaro 2016-03-19 04:23:21 CDT
For the birthday issue, your suggestion looks good at first sight.
I will check better during the week. If confirmed good I will push a commit to your name.
Comment 10 Michele Calgaro 2016-03-21 09:27:51 CDT
Just for info, I started working on it ;-)
Comment 11 Michele Calgaro 2016-03-22 20:37:42 CDT
Created attachment 2640 [details]
Patch for VCardTool::parseDateTime()

Hi Deloptes,
please try the attached patch and let me know if it is ok for you. If so, I will push to GIT.
It supports all formats specified in https://www.w3.org/TR/NOTE-datetime and the equivalent ones without '-' and ':' (so basically also 20160323T103401Z is supported)

The patch applies successfully on both R14.1 and R14.0 branches. In the end we may end up pushing it only to R14.1 since it adds functionality and change quite a bit of code. I will discuss with Slavek once you are happy with the patch.
Comment 12 Michele Calgaro 2016-03-22 20:44:24 CDT
Forgot to mention, the patch does not currently support these formats:
week:                                       2016-W11
week number + day:                2016-W11-6
progressive day from Jan 1st: 	2016-079

Do you think we need these? If so, I can add them, although it only support date in this form and not time.
Comment 13 Michele Calgaro 2016-03-22 20:53:31 CDT
>It turns out if you feed parseVCard with addresse, where birthday is empty "BDAY:\n", it would feed empty 
>string to parseDateTime, which in my opinion is not correct.
>We may need to rework this part as well via another ticket. I am not quite sure how this influences my work 
>here atm.
I also took a look at this. If the birthday string is empty or invalid, the code will set the birthday field to an empty TQDateTime object, which IMO may be acceptable (i.e. there is no birthday specified).
Did you see any specific problem with that? If not, I propose to not doing any change about it. If yes, please let me know the issue and we will look at fixing it ;-)
Comment 14 deloptes 2016-03-23 19:34:48 CDT
Hi Michele,
sorry for late response. My time window gets closed and I can follow up very slowly now.
The patch without a fix for the birthday issue is not good for me. TDE throws an error and the process supervising the execution fails. With the birthday fix it works for me.
The problem is that it only tries to set the date/time to 0. The underlay function TQDate::setYMD if I recall correctly and the other one for HMS refuse to handle the 0 value. And in fact it is true, because as we know 0 date is 1970101...
I gave up waiting for the patch because I know that I have limitted time frame to work on my project and patched myself, created deb etc etc, so its not critical anymore - at least for me.
I don't know if the BDAY is critical for other applications as I said I had to patch it to be able to use it. IMO it is OK to have it, because empty BDAY field is pretty legal.

regards
Comment 15 Michele Calgaro 2016-03-24 03:21:34 CDT
Ok, understood.
Then I will push the birthday patch to both R14.0.X and R14.1, while the regex patch will be pushed only to R14.1 for the moment. After discussing with Slavek, we may push that to R14.0.x perhaps.
Comment 16 Michele Calgaro 2016-03-24 03:37:01 CDT
Deloptes,
I will push the patch for the birthday with your name, being your work.
What name and email address should I use?
Comment 17 Michele Calgaro 2016-03-24 03:40:46 CDT
Regex patch pushed to R14.1.0 in commit f9abaf5
Comment 18 Michele Calgaro 2016-03-24 08:12:22 CDT
Regex patch also pushed to R14.0.x in commit eacbecd
Comment 19 Slávek Banko 2016-03-24 19:51:12 CDT
(In reply to Michele Calgaro from comment #18)
> Regex patch also pushed to R14.0.x in commit eacbecd

It is great! Before that due CardDAV addressbook I had in .xsession-errors flock of such reports:

TQTime::setHMS Invalid time 48:00:00.000

After the patch are now these reports away!
Guys, thank you for your good work!
Comment 20 deloptes 2016-03-25 06:01:51 CDT
Hi,
great to hear that from Slavek. Unfortunately the spare time, I have I use to chase the challenges on my goal - the sync.
Comment 21 Michele Calgaro 2016-03-25 08:12:02 CDT
Deloptes,
maybe you missed comment 16.

I would like to push the patch for the birthday with your name, being your work. What name and email address should I use?
Comment 22 deloptes 2016-03-26 07:03:58 CDT
Hi Michele,
sorry I missed the most and now found time to catch up on that. Sorry, but I get involved in new project and at home had to babysit for 1 week, which rendered the free time to just 1h/day.

Here my answers
Comment 12:
  week:                         2016-W11
  week number + day:            2016-W11-6
  progressive day from Jan 1st: 2016-079

I do not think this is needed for the card tool. I mentioned this because you used the word "properly", which means implement ISO-8601.

Comment 16:
You can use the yahoo or alternatively Name(E. Kotsev), Mail(office@fincom.at)

big thank you for pushing the changes
Comment 23 Michele Calgaro 2016-04-03 06:25:21 CDT
Sorry for the late action, I have been busy....

Birthday patch pushed in commit d10e1df (R14.1.x) and fef74a0 (R14.0.x).
Thanks deloptes!!