House of fail (we got scraped)

Any other tech-related topics
Message
Author
User avatar
SYSTEM
Posts: 1969
Joined: Sat Jul 31, 2010 1:19 am
Location: Helsinki, Finland

Re: House of fail (we got scraped)

#31 Post by SYSTEM » Sun Feb 14, 2016 12:59 am

joby_toss wrote:BUT! If someone has full access to TPFC database, would it be wise to try and change anything in our accounts now (password, etc.)? This is my question.
It would be pointless at least. They should be able to get the hash of your new password as easily as the current one.
My YouTube channel | Release date of my 13th playlist: August 24, 2020

User avatar
guinness
Posts: 4118
Joined: Mon Aug 27, 2007 2:00 am
Contact:

Re: House of fail (we got scraped)

#32 Post by guinness » Sun Feb 14, 2016 1:46 am

This is a sad topic if I must say.
Added 179 Applications: Portable

User avatar
joby_toss
Posts: 2902
Joined: Sat Feb 09, 2008 9:57 am
Location: Romania
Contact:

Re: House of fail (we got scraped)

#33 Post by joby_toss » Sun Feb 14, 2016 1:53 am

This is too freakin' scary, so I choose not to believe it at this moment. :roll:

If the unthinkable happens and TPFC goes temporarily down, how could we still stay in touch (many users aren't even listing an e-mail address in their profiles)? I'm going to say something outrageous now, so please, forgive me: I wish there was a face.book page (or something similar) for this situation (yes, the #portablefreeware IRC channel is still live on freenode, but I lost all rights to it, not sure why)...

User avatar
SYSTEM
Posts: 1969
Joined: Sat Jul 31, 2010 1:19 am
Location: Helsinki, Finland

Re: House of fail (we got scraped)

#34 Post by SYSTEM » Sun Feb 14, 2016 2:44 am

joby_toss wrote:This is too freakin' scary, so I choose not to believe it at this moment. :roll:

If the unthinkable happens and TPFC goes temporarily down, how could we still stay in touch (many users aren't even listing an e-mail address in their profiles)? I'm going to say something outrageous now, so please, forgive me: I wish there was a face.book page (or something similar) for this situation (yes, the #portablefreeware IRC channel is still live on freenode, but I lost all rights to it, not sure why)...
The reason for most of us to not show email addresses publicly is fear of spam.

If TPFC goes down, I believe we can temporarily go to PortableApps.com forums. They are closely related to us after all.

Me, in particular, you can reach via Google+. You can visit my YouTube channel (S9uareHead) that has a Google+ page.
My YouTube channel | Release date of my 13th playlist: August 24, 2020

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: House of fail (we got scraped)

#35 Post by Andrew Lee » Sun Feb 14, 2016 4:15 am

I'm trying my best to look into this.

I'd prefer not to discuss publicly the areas I'm looking into for obvious reasons until the dust has settled.

DO NOT PANIC!!! :D

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: House of fail (we got scraped)

#36 Post by Andrew Lee » Sun Feb 14, 2016 4:41 am

Guys, I am glad to report that it is indeed a proxy, probably with a content modification filter inserted to change slightly the pages it serve.

I made a stupid mistake earlier when I was checking the proxy hypothesis. When a search is performed with no results, the query is not added to the database. Because I was trying to "tag" the queries, the query keywords were ones which didn't return any results, hence the database was not modified. Once I took that into account and starting using keywords with non-zero results, the proxy hypothesis was confirmed.

It's a relief that nothing was leaked. I actually spent some time combing through the logs to ascertain that the server was not broken into. As far as I can tell, nothing suspicious was found.

Thanks to SYSTEM for coming up with the correct hypothesis! You are a genius!

@webfork: I will re-enable the backup procedure tomorrow once I run through everything again and am convinced that nothing was compromised. Thanks for your patience.

User avatar
Andrew Lee
Posts: 2606
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: House of fail (we got scraped)

#37 Post by Andrew Lee » Sun Feb 14, 2016 4:55 am

I have also blocked the proxy IP (it's sneaky, the proxy IP is different from the website IP).

Pages which have been accessed previously are cached by the proxy unfortunately, but any new access will be blocked.

User avatar
SYSTEM
Posts: 1969
Joined: Sat Jul 31, 2010 1:19 am
Location: Helsinki, Finland

Re: House of fail (we got scraped)

#38 Post by SYSTEM » Sun Feb 14, 2016 5:11 am

Andrew Lee wrote:Guys, I am glad to report that it is indeed a proxy, probably with a content modification filter inserted to change slightly the pages it serve.

I made a stupid mistake earlier when I was checking the proxy hypothesis. When a search is performed with no results, the query is not added to the database. Because I was trying to "tag" the queries, the query keywords were ones which didn't return any results, hence the database was not modified. Once I took that into account and starting using keywords with non-zero results, the proxy hypothesis was confirmed.
Whew. I'm relieved. :)
My YouTube channel | Release date of my 13th playlist: August 24, 2020

User avatar
I am Baas
Posts: 4148
Joined: Thu Aug 07, 2008 4:51 am

Re: House of fail (we got scraped)

#39 Post by I am Baas » Sun Feb 14, 2016 5:14 am

@Andrew Lee

What a relief. Thanks for looking into this + hat's off to you SYSTEM..

Let's take auqk.org down.
Bəəs 2.0

User avatar
__philippe
Posts: 624
Joined: Wed Jun 26, 2013 2:09 am

Re: House of fail (we got scraped)

#40 Post by __philippe » Sun Feb 14, 2016 5:51 am

I am Baas wrote:...Let's take auqk.org down.
Which way: Lawsuit or DDoS attack ? ... :twisted:

User avatar
joby_toss
Posts: 2902
Joined: Sat Feb 09, 2008 9:57 am
Location: Romania
Contact:

Re: House of fail (we got scraped)

#41 Post by joby_toss » Sun Feb 14, 2016 6:19 am

@Andrew: You're their oldest user, it appears. :)

Image

User avatar
Midas
Posts: 5824
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: House of fail (we got scraped)

#42 Post by Midas » Sun Feb 14, 2016 8:42 am

joby_toss wrote:If the unthinkable happens and TPFC goes temporarily down, how could we still stay in touch (many users aren't even listing an e-mail address in their profiles)? I'm going to say something outrageous now, so please, forgive me: I wish there was a face.book page (or something similar) for this situation (yes, the #portablefreeware IRC channel is still live on freenode, but I lost all rights to it, not sure why)...
  • In case it serves for anything, I took the liberty of creating the POTPFC (People Of TPFC) Slack group: http://potpfc.slack.com/...

    Anyone can join, email is needed (temporaries are OK), and approval is required. :|

User avatar
webfork
Posts: 9799
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: House of fail (we got scraped)

#43 Post by webfork » Sun Feb 14, 2016 11:01 am

joby_toss wrote:If the unthinkable happens and TPFC goes temporarily down...
We have taken some steps in the background to help make sure that the site will continue beyond any one individual. If things do drop off, it's easily possible we'll have a few days or even weeks but we should survive. In the very unlikely event that three people all in different countries who have never met die simultaneously well ... not much we can do about that.
I am Baas wrote:Thanks for looking into this + hat's off to you SYSTEM..
Ditto.
I am Baas wrote:Let's take auqk.org down.
Well it's throwing up a 403 at the moment, I guess because of the proxy steps Andrew took.
Midas wrote:I took the liberty of creating the POTPFC (People Of TPFC) Slack group: http://potpfc.slack.com/...
Wait, what is this?

User avatar
Midas
Posts: 5824
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: House of fail (we got scraped)

#44 Post by Midas » Sun Feb 14, 2016 12:10 pm

webfork wrote:
Midas wrote:I took the liberty of creating the POTPFC (People Of TPFC) Slack group: http://potpfc.slack.com/...
Wait, what is this?
  • Check it out -- Slack is a free web based IM system, getting quite popular ATM, who has apps for Android and iOS...

User avatar
webfork
Posts: 9799
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: House of fail (we got scraped)

#45 Post by webfork » Sun Feb 14, 2016 1:02 pm

Midas wrote:Check it out -- Slack is a free web based IM system, getting quite popular ATM, who has apps for Android and iOS...
I can't get in to test. PM me.

Anyway, if anyone wants to IM me off-site, I've lately been using the Tox network (i.e. qTox or Isotoxin). Tox ID: F5AD9E8EF1A0087A15E41BF275341C0A8FD205DEE1AA18D3FA848C582CC0181302E9D9221F60

Edit: I was listing an AOL account here but the client I was using evidently stopped working (InstantBird with required encryption).

Post Reply