tag:monkeytype.instatus.com,2005:/history Monkeytype Status - Incident history 2025-03-23T11:00:00.000+00:00 Monkeytype tag:monkeytype.instatus.com,2005:Maintenance/cm8jzto090002jdxcni9cg5qy 2025-03-23T11:00:00.000+00:00 2025-03-23T13:15:30.419+00:00 Server upgrade Type: Maintenance

Duration: 2 hours and 15 minutes

Affected Components: Third Party: DigitalOcean → Droplets → NYC1, , , ,

Mar 23, 13:15:30 GMT+0
Completed - Upgrade complete and login issues have been fully resolved..

Mar 23, 11:00:01 GMT+0
Identified - Maintenance is now in progress.

Mar 23, 11:00:00 GMT+0
Identified - We are planning to upgrade the server and increase the database size. Not sure exactly how long it will take..

Mar 23, 12:37:03 GMT+0
Identified - Upgrade was successful, but we are running into another issue. Fixing now..

]]> tag:monkeytype.instatus.com,2005:Incident/cm0l09vxw001rnzg0whv8xat4 2024-09-02T12:57:40.803+00:00 2024-09-02T12:57:40.803+00:00 Philippines issues Type: Incident

Duration: 24 days, 20 hours and 9 minutes

Affected Components: ,

Sep 2, 12:57:40 GMT+0
Investigating - It seems like users in the Philippines are unable to access the API server. We are trying to find out whats going on..

Sep 2, 14:59:37 GMT+0
Identified - Cloudflare seems to be the issue. Some of their Philippines based servers are having some problems. Hopefully once those servers are back up and running this issue should resolve itself..

Sep 18, 08:47:35 GMT+0
Identified - We have concluded that only users using PLDT as their internet service provider are encountering issues. We tried contacting them but no answer so far..

Oct 28, 09:06:39 GMT+0
Resolved - This incident has been resolved..

]]> tag:monkeytype.instatus.com,2005:Incident/clz6sy68j1633683hodiizevqos 2024-07-29T09:44:08.106+00:00 2024-07-29T09:44:08.106+00:00 Login issues Type: Incident

Duration: 12 minutes

Affected Components:

Jul 29, 09:44:08 GMT+0
Identified - We are continuing to work on a fix for this incident..

Jul 29, 09:55:59 GMT+0
Resolved - This incident has been resolved..

]]> tag:monkeytype.instatus.com,2005:Incident/clz1kofvo471824iholco9kgfj9 2024-07-25T17:53:45.049+00:00 2024-07-25T17:53:45.049+00:00 Server update failed Type: Incident

Affected Components:

Jul 25, 17:53:45 GMT+0
Resolved - We are back online..

]]> tag:monkeytype.instatus.com,2005:Incident/clyogdos4256574h2n4t2pwwh9c 2024-07-16T13:32:25.858+00:00 2024-07-16T13:32:25.858+00:00 Server is down Type: Incident

Affected Components: ,

Jul 16, 13:32:25 GMT+0
Resolved - We are running into some issues with the new backend update. Working on it..

]]> tag:monkeytype.instatus.com,2005:Incident/clwewki6y484839qtnaxvi3w0g4 2024-05-20T11:48:31.263+00:00 2024-05-20T11:48:31.263+00:00 Server is down Type: Incident

Affected Components:

May 20, 11:48:31 GMT+0
Resolved - The server is failing to boot due to some permission issues. Fixing now..

]]> tag:monkeytype.instatus.com,2005:Maintenance/clwb4ppu9121155bfoj2qefktlk 2024-05-18T16:00:00.000+00:00 2024-05-18T16:00:00.000+00:00 Data migration: part 3 Type: Maintenance

Duration: 4 hours and 3 minutes

Affected Components:

May 18, 16:00:00 GMT+0
Identified - Continuing the data migration (idk why we keep running into issues everytime)..

May 18, 16:00:01 GMT+0
Identified - Maintenance is now in progress.

May 18, 20:03:10 GMT+0
Completed - Migration complete..

]]> tag:monkeytype.instatus.com,2005:Maintenance/clw9hbza559240b6oi6gtrt4ml 2024-05-17T11:00:00.000+00:00 2024-05-17T16:00:00.000+00:00 Data migration: part 2 Type: Maintenance

Duration: 5 hours

Affected Components: ,

May 17, 16:00:00 GMT+0
Completed - Maintenance has completed successfully.

May 17, 11:00:00 GMT+0
Identified - Continuing Thursdays failed migration on Friday..

May 17, 11:00:01 GMT+0
Identified - Maintenance is now in progress.

]]> tag:monkeytype.instatus.com,2005:Maintenance/clw92kz9u86746bgojtf5voqss 2024-05-16T09:49:30.257+00:00 2024-05-16T16:42:17.616+00:00 Data migration Type: Maintenance

Duration: 14 minutes

Affected Components: , ,

May 16, 16:42:17 GMT+0
Completed - Maintenance over. Will continue tomorrow..

May 16, 09:49:30 GMT+0
Identified - The server will be under maintenance to perform test activity data migration..

May 16, 16:28:35 GMT+0
Identified - Ran into some issues today. Gonna finish up one last thing, put the server back online and continue tomorrow..

]]> tag:monkeytype.instatus.com,2005:Incident/clsal9laj25382bwoctuhllbcl 2024-02-06T16:42:12.300+00:00 2024-02-06T17:52:36.936+00:00 Server is down Type: Incident

Duration: 1 hour and 10 minutes

Affected Components: , ,

Feb 6, 17:52:36 GMT+0
Resolved - Everything should be awesome.

Feb 6, 16:42:12 GMT+0
Investigating - We are currently investigating this incident..

Feb 6, 16:54:22 GMT+0
Identified - Turns out that Digitalocean is running a maintenance. It was meant to not affect droplets too much, however it seems like something went wrong. <https://status.digitalocean.com/incidents/1jwy0vy3hjfb>.

Feb 6, 17:35:53 GMT+0
Monitoring - The database and Discord bot are back up, but we are running into some issues starting the API server..

]]> tag:monkeytype.instatus.com,2005:Incident/clrxjt1j5282375bnocw34tek02 2024-01-28T13:40:19.748+00:00 2024-01-28T13:40:19.748+00:00 DDOS Attack Type: Incident

Duration: 2 hours and 7 minutes

Affected Components: , ,

Jan 28, 13:40:19 GMT+0
Identified - The server got hit with a very large DDOS attack. It will remain offline for the time being while the logs are being checked and a potential counter measure is found. Sorry everyone!.

Jan 28, 15:47:36 GMT+0
Resolved - This incident has been resolved..

]]> tag:monkeytype.instatus.com,2005:Maintenance/clj4iuvyb93230c9oge27d45ix 2023-06-20T18:00:00.000+00:00 2023-06-20T18:05:17.405+00:00 Maintenance Type: Maintenance

Duration: 4 hours and 14 minutes

Affected Components: , ,

Jun 20, 18:05:17 GMT+0
Identified - Starting backup..

Jun 20, 18:00:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time to upgrade our backend server. Estimated duration is up to 3 hours, but it may complete sooner..

Jun 20, 21:57:04 GMT+0
Identified - Alright, database has been copied. Almost done, I promise!.

Jun 20, 20:27:12 GMT+0
Identified - Backup complete (it took way longer than expected so the maintenance will probably take a bit longer).

Jun 20, 22:14:15 GMT+0
Completed - Maintenance has completed successfully..

Jun 20, 21:01:02 GMT+0
Identified - Sorry everyone, looks like we jumped the gun on the previous update - it seems like the backup has not completed yet, and we are getting some weird errors from out server provider. Trying to figure out whats going on..

Jun 20, 21:13:52 GMT+0
Identified - Alright, looks like everything is back to normal (I only panicked a little). Copying the database to a larger drive now!.

Jun 20, 18:00:01 GMT+0
Identified - Maintenance is now in progress.

]]> tag:monkeytype.instatus.com,2005:Incident/cl87tzk2i8167hqmy9w3m7otl 2022-09-18T21:08:54.940+00:00 2022-09-18T21:08:54.940+00:00 DDOS Attack Type: Incident

Duration: 2 hours and 40 minutes

Affected Components:

Sep 18, 21:08:54 GMT+0
Identified - We are continuing to work on a fix for this incident..

Sep 18, 23:48:46 GMT+0
Resolved - We patched the hole and banned the offending IPs. We've disabled sign up for a bit, we will enable it in the near future.

]]> tag:monkeytype.instatus.com,2005:Maintenance/cl7opc98s6396hzoovkoiqu56 2022-09-05T13:15:00.000+00:00 2022-09-05T14:13:50.132+00:00 Maintenance Type: Maintenance

Duration: 59 minutes

Affected Components:

Sep 5, 14:13:50 GMT+0
Completed - Maintenance has completed successfully..

Sep 5, 13:15:00 GMT+0
Identified - We are planning a server maintenance to deploy a new update.

Sep 5, 13:15:01 GMT+0
Identified - Maintenance is now in progress.

]]> tag:monkeytype.instatus.com,2005:Incident/cl6bysemc36089vmloszlbuy92 2022-08-02T09:14:59.854+00:00 2022-08-02T09:14:59.854+00:00 DDOS attack Type: Incident

Affected Components: ,

Aug 2, 09:14:59 GMT+0
Resolved - We just resolved the issue!.

]]> tag:monkeytype.instatus.com,2005:Maintenance/cl4pu7vla17113j9oimcedkddw 2022-06-22T18:00:00.000+00:00 2022-06-22T20:51:04.048+00:00 Maintenance Type: Maintenance

Duration: 2 hours and 51 minutes

Affected Components: , ,

Jun 22, 20:51:04 GMT+0
Completed - Maintenance has completed successfully..

Jun 22, 18:04:46 GMT+0
Identified - Backup started.

Jun 22, 18:00:00 GMT+0
Identified - We are planning for a scheduled maintenance to upgrade the server..

Jun 22, 18:00:01 GMT+0
Identified - Maintenance is now in progress.

Jun 22, 18:00:01 GMT+0
Identified - Maintenance is now in progress.

Jun 22, 20:22:13 GMT+0
Identified - Backup complete. Starting upgrade..

]]> tag:monkeytype.instatus.com,2005:Maintenance/cl3orjrt74079lemy2ty0zvns 2022-05-28T12:00:00.000+00:00 2022-05-28T12:02:23.193+00:00 Maintenance Type: Maintenance

Duration: 2 hours and 2 minutes

Affected Components: ,

May 28, 12:02:23 GMT+0
Identified - Backup started.

May 28, 14:01:47 GMT+0
Completed - Maintenance has been completed successfully. We found 23921 duplicated names by 75945 users. Sorry about this mess, won't happen again!.

May 28, 12:00:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time. We are planning to create a full backup before running a script that will check the database for duplicated usernames after identifying issues that allowed users to register with a taken name. Users that registered with a duplicated name first get to keep the name, while the other users will have their name reset. We are very sorry for this. We are not sure how long it will take, so we are overestimating the duration by a lot. .

May 28, 12:00:01 GMT+0
Identified - Maintenance is now in progress.

May 28, 13:40:11 GMT+0
Identified - Backup complete. Running script to find duplicates..

]]> tag:monkeytype.instatus.com,2005:Maintenance/cl2uykinu153693jaoe0v7ks05r 2022-05-06T21:45:00.000+00:00 2022-05-06T23:45:00.000+00:00 Maintenance Type: Maintenance

Duration: 2 hours

Affected Components: ,

May 6, 23:45:00 GMT+0
Completed - Maintenance has completed successfully.

May 6, 21:45:00 GMT+0
Identified - We are planning a maintenance to upgrade our Discord bot. .

May 6, 21:45:01 GMT+0
Identified - Maintenance is now in progress.

]]> tag:monkeytype.instatus.com,2005:Maintenance/cl16ll3ex439222etn10b2ctbuw 2022-03-25T19:00:00.000+00:00 2022-03-25T19:00:00.000+00:00 Maintenance Type: Maintenance

Duration: 50 minutes

Affected Components: ,

Mar 25, 19:00:00 GMT+0
Identified - We are planning maintenance to deploy a new Monkeytype version..

Mar 25, 19:50:08 GMT+0
Completed - Maintenance has completed successfully..

Mar 25, 19:00:01 GMT+0
Identified - Maintenance is now in progress.

]]>