Outage - Server issue RESOLVED

1151618202133

Comments

  • carolin
    carolin Member, Beta Tester Posts: 34
    25 Agrees 10 Comments 5 Likes Name Dropper
    ✭✭
    bjlaurie said:
    I've seen many people say their V1 FBs have been experiencing this problem, mine is a V2 so it appears to be affecting both products. Last night I had enough and just unplugged my FB. But then continued to receive offline/online notifications all night long. I'm no IT expert  but that seems to point to a server problem for sure! It would be nice to hear from Fing in some way, any way. Are you out there?
    That is definitely a problem if your fingbox was kept unplugged but yet fing notified that it came back up.  Something is seriously wrong with their code for that to happen, and makes one wonder if it's reliable at ALL anymore.  It might be a single, simple line of code that is wrong, but it's a major deal for it to report your fingbox is online when it isn't even plugged in  :s 
    DanM
  • seh2000
    seh2000 IrelandMember, Beta Tester Posts: 111
    Second Anniversary 10 Comments 5 Agrees First Answer
    ✭✭✭
    @marco_from_fing
    Thanks Marco for the update appreciated.
    Perhaps similar updates to be provided if there is a next tie similar or tis happens again.
    SRPcarolin
  • Popula
    Popula Member Posts: 30
    5 Answers 10 Comments 5 Likes Name Dropper
    ✭✭

    Absolutely. New user here (1+ week) and already frustrated by the instability of the product.

    Will there be a v3 soon, by any chance? v2 is already two years old.

    And I think it needs improved WIFI6(e) mesh network compatibility.

    carolin
  • Shooter
    Shooter Member Posts: 78
    25 Agrees 25 Likes 10 Comments First Answer
    ✭✭✭
    edited March 17, 2021 #516
    @marco_from_fing,

    On behalf of many, very frustrated FingBox users, I would like to express our gratitude for at least providing this more comprehensive and direct explanation of what has been occurring for an excessively long time, especially for those of us who were first impacted months ago.

    Representing myself, I have a few questions and concerns I hope you will also be willing to address.

    1. With a date, when did this all begin? 

    2. Unless I have misunderstood, it seems that these outages, disconnects, and other issues push alerts only in the late evening/early morning hours when most are asleep, regardless of time zone. If that's correct, why only during times when FingBox owners aren't awake to respond to these alerts? Whether this phenomena is occurring based on time zones and not at the same time regardless of timezone, it seems suspiciously strange to me!

    3. "So far, we tried quite a lot of alternatives. Sometimes AWS hardware also degrades without warning. Usually AWS replaces it and moves transparently, but it wouldn't be the first time that we had to it ourselves. The re-start of the service doesn't fix but a physical reboot of the VM does, pointing out to something strange in the machine."
    Since it appears that you don't know the cause, can you assure me/us that this isn't the result of malware, whether from a directed server hack or some other source, such as from an individual FingBox or network they monitor? 

    4." We are now verifying if the root cause lies inside some specific Fingbox or network they monitor, tracing which Fingbox disconnects before the issue is raised." 
    But what if it is an individual FingBox that has been disconnected and remains disconnected from the individual network router, out of frustration or an abundance of caution, meaning it can't be isolated, but then is reconnected after the all clear is sounded, only to start the process all over again? 

    Although I'll stop with just the list above, I'm sure I'll have more questions and concerns. And I'm also sure there will be a landslide of the same from others. 

    I'm not going to place blame on anyone here, but we should have received this type of update long ago, avoiding the potential of irreparable damage, and placing the security of the individual, private, residential Wi-Fi networks that many of us non-technical types have entrusted with Fing.


    *Please let me know if you will be responding to my questions and concerns shortly, or that you won't be or can't. Thank you! 


    PopulaDalekSchorsch64carolinMarius
  • Wim
    Wim Member, Beta Tester Posts: 18
    10 Comments 5 Likes Name Dropper First Anniversary
    @marco_from_fing thanks for this elaborate explanation. I will sit still and wait. Hoping it is not my Fingbox that ruffles your fur. Sometimes it is hard work, I know, I have been there. Satisfaction will be your part when you find the cause. And a huge user appreciation!!
    PopulaSRPDavidPDX
  • Ciaran
    Ciaran Administrator Posts: 1,180
    1000 Comments 250 Likes 50 Answers 100 Awesomes
    admin
    Thanks @marco_from_fing for the update. As mentioned, we do appreciate the frustration that this issue has caused to date & we are committed to resolving it. In the meantime, I would just like to remind everybody of our Community Guidelines. Whilst we appreciate users being frustrated/annoyed at the ongoing issue, we want to ensure that all users treat each other with respect and refrain from any abusive or disrespectful posts towards others.
    I would also like to remind all users that 'Fing Moderators' are Fing Users just like everybody else here, and not direct employees of Fing. Our Moderators are a very active users that do their utmost to assist, guide & contribute on the Fing Community. Hence, any frustrations you may have in relation to this issue (or any other issue) should not be directed at our moderators. As a reminder, please see our Community Guidelines:  https://community.fing.com/discussion/19/community-guidelines-read-before-posting

    To note: we have removed some posts on this thread whereby these guidelines were contravened, we have reached out to the relevant user/s to advise of the specific action (and give the benefit of the doubt that no bad intent was meant).  If users continue to go against Community Guidelines, this may result in their account being suspended or removed.
    Thank you for your understanding in this regard.
    Ciaran (Admin at Fing)

    seh2000PopulaSchorsch64asrijojo13schmegs
  • Shooter
    Shooter Member Posts: 78
    25 Agrees 25 Likes 10 Comments First Answer
    ✭✭✭
    edited March 17, 2021 #519
    Ciaran said:
    Thanks @marco_from_fing for the update. As mentioned, we do appreciate the frustration that this issue has caused to date & we are committed to resolving it. In the meantime, I would just like to remind everybody of our Community Guidelines. Whilst we appreciate users being frustrated/annoyed at the ongoing issue, we want to ensure that all users treat each other with respect and refrain from any abusive or disrespectful posts towards others.
    I would also like to remind all users that 'Fing Moderators' are Fing Users just like everybody else here, and not direct employees of Fing. Our Moderators are a very active users that do their utmost to assist, guide & contribute on the Fing Community. Hence, any frustrations you may have in relation to this issue (or any other issue) should not be directed at our moderators. As a reminder, please see our Community Guidelines:  https://community.fing.com/discussion/19/community-guidelines-read-before-posting

    To note: we have removed some posts on this thread whereby these guidelines were contravened, we have reached out to the relevant user/s to advise of the specific action (and give the benefit of the doubt that no bad intent was meant).  If users continue to go against Community Guidelines, this may result in their account being suspended or removed.
    Thank you for your understanding in this regard.
    Please accept my sincere apology for offending any moderator! 
    seh2000CiaranSchorsch64carolin
  • SRP
    SRP LAMember Posts: 117
    100 Comments 25 Agrees 25 Likes 5 Awesomes
    ✭✭✭

    Thank you @marco_from_fing & @Ciaran for the very comprehensive update. It's clear from your post that the Fing server team has been actively exploring multiple lines of inquiry to ascertain the root cause of the issue and resolve it. I wish you well in your endeavors moving forward and appteciate the work of all involved on our behalf.

    seh2000carolin
  • marco_from_fing
    marco_from_fing Administrator, Fing Team Posts: 60
    Second Anniversary 25 Likes 10 Comments 5 Agrees
    ✭✭✭
    Thanks everybody for your patience and understanding.
    @Shooter, to provide a bit more details:
    • I think the first recorded issues were in mid February. Regarding the hours at which they have taken place, they were quite different and sometimes they happened at nighttime (GMT), ad weekend time, at early workday time, so no discernible time pattern there. Please consider also that Fingboxes are spread across multiple time zones, and their disconnection times have also been put in correlation with known outages to figure out if something external could be the cause.
    • We just recently performed a full security assessment & teardown of Fing consumer products by an external professional security company that included also Fingboxes, their connectivity and their internal workings, and no critical nor major issue have been reported. The current attempt to identify a pool of Fingboxes is also useful to figure out if the issue comes from a few specific devices.  
    • If a specific device (or rather, the data from its network) is causing the issue, we hope to analyze and solve it. In the worst case, we can also block that device from pushing data. There is already an administrative lock failsafe to exclude data from some devices for testing and for security purposes. 
    We hope to provide another update as soon as we have additional results from further analysis.
    seh2000carolinSRPSchorsch64schmegs
  • carolin
    carolin Member, Beta Tester Posts: 34
    25 Agrees 10 Comments 5 Likes Name Dropper
    ✭✭
    edited March 17, 2021 #522
    marco_from_fing  Thank you for the detailed update.  I think I speak for many here in requesting that we continue to receive updates from you or your team as you continue working on finding the issue(s) that were the root cause.  I think it would go a long way in restoring faith in fing for many who are frustrated by the numerous bounce messages from fing combined with lack of updates from the team until now.  Hopefully, you will also continue to respond to the questions posed by folks in response to your updates as well.  
    Schorsch64
  • Shooter
    Shooter Member Posts: 78
    25 Agrees 25 Likes 10 Comments First Answer
    ✭✭✭
    Thank you @marco_from_fing,

    This is real progress and greatly appreciated! While you prepare your next update, I will prepare my next round of questions accordingly.

    Until then, can you please clarify each bullet point a bit more?

    1) Is there any way to determine a precise date, since I recall having these outages as far back as this past December. At least January.

    2) So what you're saying is that Fing servers were definitely, unequivocally never compromised, including a hack? 

    3) Regarding any FingBoxes that have been, or will be blocked from pushing data, have the owners been notified or will they be notified?

    Thanks again and keep up the good work and communication! 
  • Mrite461
    Mrite461 Member, Beta Tester Posts: 4
    First Comment First Anniversary Photogenic

    Thanks for the update. My V2 fingbox is one of those devices that have been going offline/online sporadically over the everyday over the last several weeks. It’s been frustrating but I’m glad to know there is work being done to figure out what’s going on. Hopefully a solution is found soon.

  • carolin
    carolin Member, Beta Tester Posts: 34
    25 Agrees 10 Comments 5 Likes Name Dropper
    ✭✭
    edited March 17, 2021 #525
    It was quiet last night and all of today, until 7:12pm EDT.  I just got the email saying my V1 fingbox is offline, yet the lights are still the normal green, never changed color.  I hope this doesn't mean we're in for another round of bouncing fing messages tonight.  I'm in FL, USA. 4 minutes later, it said the fingbox was back up.  The color on the fingbox never changed to orange, it stayed solid normal green color.
  • CMorning
    CMorning Member Posts: 2
    Photogenic First Comment

    Same here

  • SRP
    SRP LAMember Posts: 117
    100 Comments 25 Agrees 25 Likes 5 Awesomes
    ✭✭✭
    edited March 17, 2021 #527

    Ditto for me. Last night clear. Disconnecting/reconnecting as of 4.14pm PST today. But V2 blue glow for me. :)

  • Dalek
    Dalek Member Posts: 61
    25 Agrees 10 Comments First Anniversary 5 Awesomes
    ✭✭✭
    edited March 17, 2021 #528

    My v1FB has “Alert me when state changes” set to Off and “State change timeout” set to 24 hours, yet at 4:13 pm Pacific Daylight Time I got a “Fing Alert” email notification saying my FB went offline and 6 minutes later at 4:19 pm PDT I got another saying my FB is back online.

    Just got another one at 4:32 pm saying it’s offline.

    WTF? I should NOT be getting these email notifications and besides 6 minutes is way less than 24 hours!

    carolinMaggieL
  • FingCatLord
    FingCatLord Member, Beta Tester Posts: 17
    10 Comments 5 Likes First Anniversary Photogenic
    ✭✭
    Well everything was quiet for all of today, but I just started the whole back and forth between offline and online alerts.  My FB has not gone into orange light spin mode and have had no LED change, still steady, but the offline and online alerts seemed to have returned for me also.  Plus having a couple issues with the App as in sync and loading screens.  And as I type this, just went through my 4th Offline/Online alert cycle, guess its not fixed yet, too bad I was really thinking it was resolved.  I am guessing everyone else is probably seeing the same, as a couple of you have already chimed in with the same results that I have.
    carolin
  • max2198
    max2198 Member Posts: 20
    10 Comments First Anniversary Photogenic
    I am also getting Offline/Online alerts. also notice some features are not working  under Network the follwing  Events, Wi-Fi speed test I get "Could not start", Bandwidth Analysis, I get "Could not start"   Router vulnerability, find hidden cameras .   under  people: digital fence "startup failure", Under internet:  Internet speed test "startup falure". 
        I do get notifications when devices enter and leave my network  
    FingCatLordShootercarolin
  • Matt_T
    Matt_T Member Posts: 9
    Photogenic Name Dropper First Comment
    This afternoon...after things seemed to be working fine for a few days...I'm getting intermittent "Cannot find the selected network in your account" for both my FingBoxes (Gen 1 and Gen 2).  Fingbox 1 is now also flashing orange. 
    Shooter
  • King
    King Member Posts: 40
    10 Comments 5 Agrees Name Dropper First Anniversary
    ✭✭

    Ever since the most recent firmware updates my Fingbox keeps going offline. I have changed the network cable with no effect. I have a Fingbox v1.

  • Matt_T
    Matt_T Member Posts: 9
    Photogenic Name Dropper First Comment
    Note that, based on the last outage, flashing orange indicates an issue with the Fing server (and not my ISP connection...as obviously I have access to the Internet to submit this message!). 

    ShootercarolinKrB
  • carolin
    carolin Member, Beta Tester Posts: 34
    25 Agrees 10 Comments 5 Likes Name Dropper
    ✭✭
    Dalek said:

    My v1FB has “Alert me when state changes” set to Off and “State change timeout” set to 24 hours, yet at 4:13 pm Pacific Daylight Time I got a “Fing Alert” email notification saying my FB went offline and 6 minutes later at 4:19 pm PDT I got another saying my FB is back online.

    Just got another one at 4:32 pm saying it’s offline.

    WTF? I should NOT be getting these email notifications and besides 6 minutes is way less than 24 hours!

    I've noticed that fing ignores having the alert set to off as well for my v1 fingbox.  I also have it set to "off" but still get alerted.  
    ShooterFingCatLordMaggieL
  • AustinJerry
    AustinJerry Member Posts: 101
    25 Likes 10 Comments Name Dropper 5 Awesomes
    ✭✭✭
    The behavior is different for me today.  I received a Fingbox Offline alert several hours ago, and since then I have received at least four alerts that it is back online.
    Fingbox owner from the beginning
  • Piman
    Piman Member Posts: 0
    My fingbox is constantly cycling off and online.  Most times I don’t notice any service interruptions.  Anything I can try?
  • BK303
    BK303 Member Posts: 23
    10 Comments Name Dropper 5 Likes First Anniversary
    ✭✭
    #537 Accepted Answer

    False offline/online notifications is widespread problem for many fingbox users right now. Please see this update from Fing team today in top threads on this issue:

    https://community.fing.com/discussion/comment/27130#Comment_27130

    MaggieL
  • Fingbean
    Fingbean Member, Beta Tester Posts: 8
    First Anniversary Photogenic First Comment
    Been having Fing "Offline" outages almost daily and in frequent spats since the beginning of March. What is going on? I wanted to try checking my fing firmware version, but it doesn't give me a firmware version number. (Just says Fingbox with a MAC underneath it) Also noticed that it will periodically display "Could not update from fingbox" at the bottom of the screen for the application. (I am checking with my iphone application version) Have tried power cycling the fingbox, but get the same results. Any help or suggestions would be much appreciated.
    Shooter
  • King
    King Member Posts: 40
    10 Comments 5 Agrees Name Dropper First Anniversary
    ✭✭

    I knew it would come to this. First the move to Lansweeper and then the push for software based services. It is all a matter of time before they eliminate the Fingbox all together. You see Fingbox users are now causing the company to spend money on hardware infrastructure with upgrades to AWS. This can be a significant cost and in the long run will push the Fingbox to be an EOL device. Get ready for more offline / online alerts and customers get frustrated and jump ship. This is so 😥...

    Shooter
  • BK303
    BK303 Member Posts: 23
    10 Comments Name Dropper 5 Likes First Anniversary
    ✭✭

    @Dalek

    In the Fing App (Android for me) from home page > bottom right "Account" to change screen >. "Notification settings" > change "send email alert as" to "Do not send" is the method I used to stop emails about fingbox offline/online.

    Those settings you mentioned under the Fingbox device screens for "Alert me when state changes" and "State change timeout" are per network device setting found under each device entry. I don't think they apply to the response generated by this issue of the entire Fingbox spuriously going offline/online.

    Hope this helps you turn off emails generated by this issue.

    carolinFingbeanRobAbram
  • King
    King Member Posts: 40
    10 Comments 5 Agrees Name Dropper First Anniversary
    ✭✭

    Thank you for the information

This discussion has been closed.