Please post all forum bugs here

granatnik

Akiba Citizen
Sep 19, 2012
3,812
4,752
Maybe calling the guy directly might help... @tliu223, hey!

(I see "Ignore" button on the hover popup.)
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
We lost our servers in Asia.
Expect a very slow site, since we have to route everything through Europe (and they will be overloaded for 100%)

Sorry about that. Not sure, why they are down. I hope @chompy can figure this out, soon.

Edit:
Systems are back up running.
Sorry about that.
DNS will take a bit to get the new systems back online.
 
Last edited:

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
sometimes it always happens at the same time.

as you may know, we are hosting on a cloud.
but a cloud is only some remote system - and the node where our main-database was hosted was shutdown about 11:45pm UTC.
about 10 minutes after that, the database-vm came back up online - but I was sleeping, since I really need to sleep sometimes

For security reasons a "normal" start is not possible - i need to manually enter a Password to decrypt the files. (better safe than sorry | also better save then sorry)
As you may see now - they system is back up online.
perhaps we should consider adding an additional master-database-node, at a different location, if something like this happens again.
But this will mean, that I have to change the whole cluster setup - again some downtime is needed :(
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
We lost our last Hoster for EU/US/everything-not-asia

expect slow page-loading (after the DNS is updated for everyone)

also ... we are again on the lookout for new servers :/
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
we got a replacement!

Bildschirmfoto 2021-12-18 um 18.06.31.png

were you affected of the fuckup I did now? (the first drop down to 0Mb/s is what I mean)
messing with the DNS and also double-checking everything will still lead to a strange replaced : with _ which I didn't find the the first time I checked.

sorry about that.
New servers are already receiving a "little" bit of traffic :)
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
asia frontend is down :/

see: https://status.akiba-online.com/status

If this won't be fixed in an hour or so, i will reroute all the asia traffic through emea

(I should really automate this already)
Bildschirmfoto 2022-01-03 um 11.57.21.png


edit
changed to emea only -> traffic went up again ^^''
Bildschirmfoto 2022-01-03 um 12.22.55.png
I will need more different locations, if this continues to happen :/


edit2
Systems are back online. DNS is changed back to full load-balancing
 
Last edited:

granatnik

Akiba Citizen
Sep 19, 2012
3,812
4,752
Good question! I recall something about this section being problematic for randomness of images or external exposure and thus everything is deleted anyway, but confirmation is warranted.
 

CoolKevin

Nutcase on the loose
Staff member
Super Moderator
Mar 30, 2007
9,994
3,594
Media Section only has 1 picture. How to gain access?
Good question! I recall something about this section being problematic for randomness of images or external exposure and thus everything is deleted anyway, but confirmation is warranted.

not sure what happened there, since the upgrade there are many things that is wrong
 
  • Like
Reactions: granatnik

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
unfortunately the media gallery did get broken when we upgraded from xenforo 1.x to 2.x
i tried to bring it back - but it was a huge mess (new database structure, new storage path, new storage format of the files.

After a "good guess" and multiple tests, I somehow ignored it (it was about 3GB of images; and they should still linger around on some backup, but it's just a pain to reformat the attachment-files to it's original name and then upload them again :/

Sorry about that.
 

diaex

New Member
Jan 26, 2022
1
0
unfortunately the media gallery did get broken when we upgraded from xenforo 1.x to 2.x
i tried to bring it back - but it was a huge mess (new database structure, new storage path, new storage format of the files.

After a "good guess" and multiple tests, I somehow ignored it (it was about 3GB of images; and they should still linger around on some backup, but it's just a pain to reformat the attachment-files to it's original name and then upload them again :/

Sorry about that.
Website is incredibly slow right now. :(
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
some severe errors with our asia servers.

one asia server is able to reach part of the load-balancer, but not the other one
the 2nd asia server doesn't reach any of the two systems

Routed everything through europe :/

I hope this is just some "minor" BGP has fucked up thing :/
 
Last edited:

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
routes are back!
Asia Servers are back to the DNS system.

Let's hope it won't break again in the next few days, because i won't be able to access the internet with more than edge-speeds.
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
there could be the possibility that the database switched to read-only mode

i'm not sure why -.-#
sorry about that.
everything is back online
 

granatnik

Akiba Citizen
Sep 19, 2012
3,812
4,752
What is going on? First I was logged out what did not happen in months or even years, then site has started losing settings?
 

granatnik

Akiba Citizen
Sep 19, 2012
3,812
4,752
Am I really so old that even my cookies expire?
 

C00Lzero

Administrator
Conqueror
Aug 14, 2010
555
1,171
Nice one.
first we have a problem with the mail-servers.
now our database-cluster crashed :/

main-system is back online - but we are missing replication & backup at the moment.
Expect another downtime to rebuild the cluster :/