MabiPro: The Authentic Old-School Mabinogi Experience

hot time Valentine's Hot Time Event 2018!
Posted on 02-13-18, 09:04 am by Arisa

What? Love is in the air and you don't have a gift prepared for your partner? Or are you just still looking for that special someone? Do not worry! We love you players here at MabiPro, and we didn't forget to get you a gift! It's simple, all you have to do is log in during the times below and you'll be able to receive special gifts. Easy, right?

Event Details
Be ingame during the following times:
• Wednesday, February 14th, 12:00PM EST
• Wednesday, February 14th, 4:00PM EST
• Wednesday, February 14th, 9:00PM EST
• Thursday, February 15th, 12:00AM EST

Don't miss out! Spreading lots of love and hearts and chocolate.

1 comment (by HenTie). Log in to post a comment.

game feature Spray System
Posted on 02-07-18, 07:36 am by Flipend0

Flippy here!
I coded something fun, interesting, and unique to MabiPro, and I personally want to see how this turns out.
Using the Decal System via creative methods, I was able to create a Spray system with custom textures.
How it works is simple:
Use the spray item and it will spray a texture on the floor.
The spray system is a work in progress, and if all goes well it may be planned to have future events to enhance this system.

Feature Details:

• Globally, only 100 Sprays are allowed on the floor.
  
• Sprays are globally erased every 36 minutes in real time.
  
• Sprays #0 and #1 will temporarily be in Shyla's Shop. Future Patches may remove this for a better future distribution system on sprays.
  
• The Spray system is a work in progress, and may change based on new ideas, functionality, and/ or system limitations.
  

2 comments (last by Mihoshika). Log in to post a comment.

event MabiPro's 1st Anniversary Event
Posted on 02-07-18, 07:23 am by Drahan

Greetings, Milletians!

On February 8th of last year, the MabiPro team made this thread: news Mabinogi Professional officially released!
We are excited to announce that today marks our 1st year anniversary of starting MabiPro.
We'd like to thank each and every one of our players who have dedicated their time to play on our server.

---

Over the period of 1 year, there has been a lot of drastic changes to our player base, development environment, ideas, and the team members.
We've encountered many problems along the way, but we've been consistently fixing whatever's thrown at us.
We'd like to thank you for staying along for the ride and motivating us to do what we enjoy: running the server.

We've decided to run a fun event to celebrate.

---

Rumor has it that if you give Shyla some special cards, she'll craft you a special dungeon pass.
She needs 5 of each M,A,B,I,P,R,O cards, they drop from monsters during the event.

Collect all 35 of them, and trade them with Shyla...then prepare for the dungeon.
The dungeon starts easy, but don't get too comfortable.
Make sure you bring a friend, as it'll require 2 or more players to enter the dungeon.

Event Details:

• This event will run from 2/7/18 to 2/14/18 (Subject to re-evaluation)
• Collect M,A,B,I,P,R,O tickets from defeating monsters
• After collecting 5 of each M,A,B,I,P,R,O tickets, turn them in to Shyla at Tir Chonaill for a trade offer of her special dungeon pass
• It requires 2 or more people on the party to be able to enter the dungeon

8 comments (last by Alder The LoreKeeper). Log in to post a comment.

statistics news MabiPro Performance Analytics now public!
Posted on 02-04-18, 10:52 am by Drahan

Hello Milletians!

If you have ever wondered any of these things:
"Man, what in the world just caused that lag spike?",
"Is the server lagging...or am I lagging?",
"I wonder how much RAM and CPU MabiPro uses at any given time...?"

then, MabiPro has the answer....public access to our internal performance analytics system!
We decided that it wouldn't be a bad idea have access to this, since it's all read-only.
I'm sure lots of you are curious about MabiPro's performance analytics.

---

You can view the analytics at http://analytics.mabinogi.ir/

---

This contains lots of cool statistics, such as:

• Player count
• Latency overhead for HA nodes
• Server RAM usage
• Disk space availability
• Bandwidth usage
• CPU usage/load

and potentially much more.

---

If we are encountering server issues, you can expect to see the reason for why appear on these graphs.
We hope you enjoy.

- Drahan

2 comments (last by sbs4lmp). Log in to post a comment.

info Recent Server Instability
Posted on 01-18-18, 02:01 am by Drahan

Hello Milletians!

Recently, you may have noticed that recently the server has had a steep decline in stability.
We take server stability very seriously, and it's a real shame that we aren't able to fulfill our goals for stability.
We know these crashes are extremely frustrating for our players, especially those who lose items during the crashes. Trust me, they are extremely frustrating for us too. It seems like every time I look away for any extended period of time, something has gone wrong.

Today, I am going to write about the each individual crash, why they happened, and how the team is planning to solve these problems in the future.

Server crash explanations
In the past 2 weeks, the server has crashed 5 times. This means, during a working day there is a 50% chance that the server will crash.
A 50% crash rate is certainly not something we are proud of.

Here is a list of the last 5 crashes, and their causes.
1: Unexpected server maintenance (to patch Spectre and Meltdown). Our server host did not notify us in advance about this.
2: Database server stalled due to resource exhaustion.
3: Login server stalled due to resource exhaustion.
4: Database server stalled due to resource exhaustion.
5: Unexpected Windows OS level error, probably caused by resource exhaustion but not verified.

---

Pet lag (or rather, the sinister database stall)
You may have experienced database server stalling first hand - it's what is known as "pet lag". This "pet lag" is much more sinister than it may look at first glance.
The reason people call it pet lag is because summon times take an extremely long time to finish, the cause of this is the database server failing to respond to queries in a timely manner.
This lag also affects logging in, saving bank info, changing channels, and basically anything that touches the database.
The game server has a very aggressive cache on player data, so you don't notice it while playing until you summon a pet usually.
If pet lag is preset, the server is on a downward spiral towards an imminent full crash - where you will not be able to play at all.

Luckily, pet lag takes a long time to fully manifest into a full crash, so you may notice we do not restart the server until it gets extremely bad (and usually right before the full crash).
Unfortunately, this causes players to have a less than desirable experience - but it's our only choice. If we immediately restart the server every time the database starts to stall, we would be restarting the server every few days. We choose the lesser of two evils.

In the event of a full database server crash, you will be unable to login (it will stay on the login screen for a very long time and never finish).
Additionally, all unsaved player data is lost permanently and rolled back to a previous state.

Login server stalling
This is simply caused by extreme resource exhaustion. Mabinogi services do not recover well from failure, and under immense pressure, servers will occasionally lose connectivity with eachother.
The moment the login server loses connectivity with the server coordinator or database server is when it's stalled until manual reboot.

You can tell this has happened when you attempt to authenticate and you're immediately met with a "Unable to connect to server" error. The login server will not accept new connections when stalled.

Windows errors
We're using Windows 2003 for the game server. Not much more to say here. The software is outdated and occasionally misbehaves, especially under high load.
We use this because the Mabinogi server we use (G13 specifically) was designed for Win 2003 - it isn't very stable in other versions of Windows. Just like the Mabinogi client isn't very compatible with Win8/Win10, the server's got the exact same problems (random freezes, etc)!

Again, we choose the lesser of two evils - unstable server software or unstable operating system. It just so happens that the operating system as a whole crashes less than unstable server software.

This particular problem, though rare, is actually relatively simple to solve (in theory) compared to the other problems: just use a more stable windows version.
We are able to manually fix each crash we encounter on updated versions of Windows server, which we have done in beta testing - but we're never fully confident in that work, never really sure that we haven't missed some small case in the hundreds of thousands of lines of assembly code that we have to work with to fix such problems.

---

A common theme

You may have noticed a common theme by now..resource exhaustion. 4/5 of our server crashes were from resource exhaustion.
You're probably thinking, "Drahan, why exactly does MabiPro suffer from resource exhaustion in the first place?".

It's a good question - especially when you know how the Mabinogi server scales. It does not scale by player count or NPC count...it scales by NPC client count.
More players does not scale linearly to more resource usage.
Our actual resource usage, for the most part, is entirely static. It almost never changes.
Under this logic, there is absolutely no reason that we should be suffering from resource exhaustion in the first place..

So why?
It all boils down to one thing: the way we host and finance the server.
MabiPro is actually under an extremely low budget, and as such, we cut corners financially...a LOT.
The biggest corner we cut is our dedicated server hosting.
To cut dedicated server costs, we've got connections with a server hosting coalition that shall not be named; and we cut a deal with them.

Not only do we get an extreme discount for dedicated shares of their servers (much cheaper than what we can buy publicly), we are also allowed to use shares of the server that are unused, as in not being used by paying customers currently, free of charge.

We're using these unused server shares to power almost 80-90% of our server.

(Those of who you are reading who are knowledgeable in the server hosting industry - no, we're not technically using a dedicated server, it's just easier to explain it that way)

What's the catch?
You see, nothing is really free in this world. We save a lot of money from this deal, but it comes at a cost: server stability.

The problem is when a paying customer decides that they want to use the "unused" resources (read as, the resources MabiPro is using) that they deserve.
These resources are ganked away from MabiPro instantly, and allocated to that paying customer until they're done using it.
After they're done using it, we gain those resources again.

(For those with knowledgeable in the server hosting industry - yes, this is a form of overselling. It isn't good.)

However, Mabinogi does not respond kindly to it's resources being forcefully taken away.
This process will instantly make MabiPro crash for one reason or another.

Progression of the problem...
The more paying customers grow, the more often we get our resources (that we aren't paying for) forcefully taken away.
In the recent weeks, this has grown to a point where it may happen every other day due to simple growth of the server host.
In other words: we need to solve this problem, and we need to solve it fast.

---

The Solution

Really, the solution to a large majority of our problems is so simple, any of you could figure it out.
We just need to rent a new dedicated server, really.
This is what we plan to do in near future.

This will increase our operational cost by a significant amount, and for the most part we pay for the server out of pocket.
Fortunately, we have a good sum of Bitcoin donations (thanks to our generous community) that we have kept for a long while - we're going to use these funds to pay for the extra expenses incurred by renting a new server.
If we decide to go through with this plan, we will need continued support from the community in the long run.

---

We will try our best not to let you guys down, as long as you don't let us down.
Hopefully you enjoyed my large wall of text.

-GM Drahan

4 comments (last by Drahan). Log in to post a comment.