It, kinda, sucks (merged contacts and other problems)

This message was edited 1 time. Last update was at 1 Sep 2010 14:54 PDT

I use Nuevasync with Maemo MfE for half an year or so. Well, actually i had problems all the way: mysteriously disappearing contacts, contacts out of sync, sudden duplicates, dropped fields and i guess almost any problem one could imagine. But yesterday.. *BANG* my address book exploded! All merged contacts suddenly unmerged, several were dropped, half of the avatars are missing.. I got some backups, so i spent half of the evening trying to repair it and.. at the morning it happened again. And.. today i was "lucky" enough to see that in progress: *suddenly* contacts count dropped from 900+ to 100-smth, then sync indicator went green and contacts started appearing back. All merged icq contacts were unmerged, as well as part of xmpp ones, but not all of them. Half of the userpics is missing again and i have no idea which half. Several contacts disappeared, though i was pretty sure they were present both on the Google and phone side. Hermes-provided profile information is partially missing as well. Contacts count on Google and on the phone still differs much.

UPD: damn, again and again. It is just broken, i am going to disable sync until the issue is resolved.

Well, to clarify the source of the suckage: the thing that sucks is the software on the N900

It's hard to be sure, because we've never been able to trigger this behavior in the lab, but the working theory is that the N900 contact merging code can't cope properly with a resync, or at least there are situations in which it can't cope.

You have the free service, so unfortunately we don't have any device history to look at (users with premium service have a tracing facility in the service that allows us to look back retrospectively to analyze strange device behavior).

If we can get this explosion effect reproduced on our test devices, I'd be happy to file a bug with the details for Nokia, but so far we haven't been able to reproduce the behavior.

I have noticed a couple of users with errors coming from Google for their contacts API today, so that may have been the trigger in this case (repeated Google errors can under some conditions cause a device resync, which should by itself be benign).

Looks very strange. It appears that for no apparent reason it tries to perform full resync and is doing it awfully wrong way:

1) erasing all contacts except "server-side" ones for IM accounts
2) sucking it back from the google

..just dropping all merge information on the floor.

The funniest part is, year ago i used different device and OS (e90) and different sync service (Seven). But the creeping problems like dropped fields, dropped and duplicate contacts, lost authorizations for IM roster etc were exactly the same. But it never "exploded" at least (actually it just could not as there was no "merged contacts" concept which involved server-side information).

The problem of devices not properly handling the 'join' between server-sourced data and locally sourced data is unfortunately quite common. The N900 is just the worst offender, partly I suspect because it has the most ways to join. The most simple example is custom ring tones : most phones will forget these when they resync. Palm's WebOS implementation however is very good. YMMV.

For this and other reasons, we take extreme measures to avoid resyncing but there are cases where we just can't prevent it (usually involving peculiar error conditions generated by the data source, Google in this case).

So the bottom line is, there is nothing either you or me can do about that?

I could upgrade to premium service (despite the fact i actually do not need any premium features), but the problem is it is unlikely to help.

I seem to be having this same problem too.

I have an N900 and in the past few days (let's call it a week), I've had about 3-4 full contact resyncs. It's only happened very rarely in the past.

I do have a premium account, so can you check to see what's going on?

As far as I can tell, I've not been doing anything different recently.

This message was edited 3 times. Last update was at 2 Sep 2010 10:58 PDT

Very same problem here! But it happened to me before. I thought it wouldn't be such a hassle using the 'duplicate contacts' feature on my N900. But some contacts are updated on Google correctly others just DELETED on Google? And you can never tell wich ones. Same if you merge contacts manually. Sometimes updated, sometimes just deleted on Google. No consitancy in wheter you use one or the other contact version (google or additional N900 Info like facebook chat, skype) if deleted or not... it is more than annoying, starts making the service unusable... A shame, I've been quite happy for a while, but it seems to get worse!

Ok, I get it, Nokia is the problem, and I even believe it. It is so important to me, that I would have joined premium as well, to get this solved. But if we have to rely on Nokia or the Maemo community to solve this, I'm afraid we might as well forget it. But if you from Nuevasync could at least tell them in more detail what their problem might be, we might have a small chance.
If I understood you correctly, these problems should occur with a N900 not matter what Exchange Service/Server one uses, right? I'm wondering, whether we could somehow proof that!?

And could the newly introduced custom fields in google not be of any help? Check also:
http://forum.nuevasync.com/forum/posts/list/0/525.page#3163

Cheers

H

Oh and one last thing: Thanks for replying so fast even to 'nonpremium' users!!!

dwels wrote:
I have an N900 and in the past few days (let's call it a week), I've had about 3-4 full contact resyncs. It's only happened very rarely in the past.

I do have a premium account, so can you check to see what's going on?

As far as I can tell, I've not been doing anything different recently.

Hi, thanks for posting because this really helped. We were able to do a detailed analysis on the problem using log data from your account.

Work is still ongoing, but it appears to be a situation where three separate things got something wrong all at once: the device, our code, and Google.
First, Google's contact API 'blew up' in that it stopped accepting requests, bouncing them with a 'temporary failure, please try later'.
Second, our code (or perhaps the Google-written library code we're using, not sure which yet), translated that temporary failure into a 'hard' failure code.
Third, the device decided to resync within less than a minute of seeing the error code (most devices re-try a few times, over a longer period before taking this route).
Fourth, the N900 exploded the merging on a resync, which it really shouldn't do.

We're working on a code change that will process this type of Google error in the way we intend to handle temporary errors rather than as a permanent error.

helgeklemm wrote:Very same problem here! But it happened to me before. I thought it wouldn't be such a hassle using the 'duplicate contacts' feature on my N900. But some contacts are updated on Google correctly others just DELETED on Google? And you can never tell wich ones. Same if you merge contacts manually. Sometimes updated, sometimes just deleted on Google. No consitancy in wheter you use one or the other contact version (google or additional N900 Info like facebook chat, skype) if deleted or not... it is more than annoying, starts making the service unusable... A shame, I've been quite happy for a while, but it seems to get worse!

Hi, we're working on a 'contacts lock' feature, at the request of N900 users. This would prevent the N900 from messing with your Google contacts in any way (hence it would have prevented the deletions you mentioned above). For premium users (sorry), we do have the contacts capture feature turned on by default. This stores your contacts data in our servers and makes it available to download via the web site as a vcard file if something unpleasant happens (like a device deleting contacts).

Thanks again for the quick response! At the Moment, I do something similar with Evolution, which is also synced with my Google Adressbook. I Copy all Contacts once in a while to a local folder, and especially before I do a dangerous sync, so that I have a backup.
The Problem is, that Evolution doesn't hold all the information, and I belive Google doesn't either. So if I copy a mistakenly deleted contact back to Google (mostly the ones with IM Info which is not shown on Google), the next sync will again leave me with a duplicate on my N900. And then if I merge the contacts on the N900 again, the are mostly deleted again on Google.
The only way out of this is to manually enter the IM Information on the N900 in the contact coming from Google and deleting the one that was left (alone) on the N900. Then you're ok, but mostly only for a few days, before the very same contacts are being a hassle again.

Could at least the last part be prevented with the solution you offer to premium users?

And: I use this service heavily, but just need the free stuff. I'd be really willing to become premium, if this would help solving the issue, but onestly I'm afraid, that my problems can't be solved by you/nuevasync. Even if you got tons of log files, because it might have something (or everything) to do with the N900. My only hope would be the 'custom' fields in Google, that I mentioned in several other posts. Couldn't they be used for a workaround on nuevasyncs side? So that at least, I don't have to manually enter all the IM Info again?

Cheers, your work is really appreciated

dboreham wrote:
Hi, thanks for posting because this really helped. We were able to do a detailed analysis on the problem using log data from your account.

Glad to help

dboreham wrote:
We're working on a code change that will process this type of Google error in the way we intend to handle temporary errors rather than as a permanent error.

Excellent. Please let me know when there's an update (so I can keep an eye out to check it works) or you hit a brick wall.
Cheers.

Folks, the fix to address this issue was deployed around 20h ago.
The change detects a previously un-known Google error and treats it as temporary, which should avoid resyncs by resync-happy devices like the N900 (most of the time).

Well, there are still 790 contacts on GMail vs 880 on the phone but at least it does not explode anymore (yet. i enabled sync again

This message was edited 1 time. Last update was at 4 Sep 2010 06:38 PDT

..and it becomes clear why it does not break things anymore: it just doesn't sync

I tried creating a contact GMail side and phone side, neither was synced. Quite surprising, as sync statistics shows that a contact was added.

UPD: seems to be temporary lapse, now it works.

Note that with the N900 there will often be a higher contact count displayed on the phone vs. Google because it includes contacts from other sources (some of which may not merge).