Open Sharing About Issues With Spot

I want to share openly about a number of problems we are facing w the spot product at @JupiterExchange, what we are doing about it, and commit to a more open and public approach to solving problems together.

First off, we actually spend a crazy amount of effort trying to protect the user and ecosystem, but sometimes to the demerit of the user experience.

For example, we worked hard on a number of automated (dynamic) slippage / gas calculations, and max gas settings to help the user execute on the lowest possible settings, and prevent raising the floor for others.

This approach works well in regular times, but in volatile times, it leads to situation where the user gets incredibly frustrated about txns and failures.

This was the case over the last few days, with the case of Monday compounded by an error in gas estimation.

We are working on few things right now, including better heuristics for dynamic slippage, a yolo mode to easily switch, and alert if max gas is too low in cases of high congestion. We will roll these out very soon, but if anyone has any thoughts or ideas, I would love to hear from them as well since this is a point of massive concern (and often heated debate) within the team.

The other thing that we do is maintain a free tier quote api that is very widely used (which we spend a ton on), but currently abused severely by people spinning up new machines to bypass rate limits, and we are stuck between an endless game of increasing the servers or face possible service degradation for our own users!

In this case, we might have to severely limit the free quote api and have an access key based access to the api instead.

One last thing is that itā€™s impossible to provide the same level of UX as a bot from user interface improvements, which means that the inevitable path will be to have bot based execution for any semi competitive swaps, which is of course what @ApeJupiter and other automated buying systems do.

Anyhow, we deeply appreciate all our users and we do a lot to keep everyone as safe and optimized as we possibly can. But some of these measures might have been counter productive, creating a number of user unhappiness we see today. We will persist and work though them, and all feedback, good or bad is incredibly valuable as we work through all these issues.

PS: Iā€™ll be opening up similar threads for the main things we are working on, including LFG, ape and other services. Looking forward to chatting w everyone.

  • For convenience, we calculate the gas required for execution on the frontend, but there was an issue that caused us to miscalculate priority fee estimation. This bug did not manifest in regular times, but became a major issue during peroids of heavy congestion. Totally our bad.

  • For DCA, we had a downtime for about 6 hours due to a CU estimation error. This caused massive inconvenience to users, causing many orders to not fill. We are incredibly sorry for this - and we will be reviewing our ops to be much more alert to these.

  • To ensure safer execution with DCA, we also do a price and slippage checks before executing the trade. However, for low liquidity tokens, the mark price and the execution price can differ dramatically, causing the engine not to execute the trade. When this safety is constanty triggered, but the information is not shown on the UI, it can cause a lot of frustration for users who thinks the engine is not running.

  • For example, the usual mark price for a token in an exchange is taken as the last trade or the current quote price in an AMM. This works well for tokens w robust liquidity (like SOL), where the price for a $1 trade will be similar to a $1000 trade. However, for a low liquidity token, both the mark price and actual execution price can differ rather drastically, causing the keeper to not execute the trade to protect against bad rates.

  • To help this issue, we will be putting in more information in the ui, as well as having smarter safety checks, vs a one size fits all approach to safety checking. These wil be implemented over time incrementally.

Apart from these, I am aware there have been ongoing issues with limit orders, price api and very long tail token discovery - i will explain all these issues in a later message and our plans to fix them.

Next week, we will do a ā€œreal talkā€ planetary call, where we will discuss our progress and challenges across a range of fronts, including team, focus, product, community and token alignment.

In the meantime, I will try to do a space or discord vc over the weekend as well to share more about whatā€™s going on and happening.

Once again, we are incredibly grateful for every single one of our users and community members, and i am sorry for any technical and communication lapses. we will make both the platform and space as good as possible, and communicate to the best of our ability.

11 Likes

I love the transparency and honesty here, only if you face the challenges you can improve and elevate.

I personally used the spot with dynamic slippage but also set on gas.

I felt it was rather the right amount because sometimes it stopped me on slippage and I had the choice to manually adapt and raise when I wanted.

So for me it was a rather expected and protected experience, I felt really in control there.

4 Likes

Good job, transparency is the key.

4 Likes

Great job mate,
Stuff happens, but Iā€™m sure we learn from these and got to test The exchange and itā€™s most vulnerable moments

3 Likes

Key thing is that problems have been identified, team tirelessly working to fix it & above all you & the team are open & transparent about it. Not much more a community can ask for. We trust in your ability to address/resolve this issue. We are with youšŸ™šŸæ

2 Likes

Just like Solana, things break and we improve on them. Thanks for your openness and honesty.

3 Likes

It shows clarity, honesty, responsibility, and openness to dialogue.

3 Likes

Love for the C.A.T. this post definitely has it all!

3 Likes

@meow just posted this post above on X to clarify some misunderstanding around Jup.ag collecting fees. Itā€™s not true that jup.ag collects fees. jup.ag provides this service for free. Also see clarification below from @solscanofficial on X how these fees are collected by 3rd party not jup.ag.

3 Likes

Please see a few important tweets I came across this morning from @meow regarding upcoming tweeter space with representatives from @bitgo & @bitglobal regarding upcoming changes to WBTC:

Another important tweet from @0xSoju on how Jupiter API have made over $100M in fees & how to invite friends using the new referral program:

3 Likes

I want to support and say kind words.
You do a great job, you are very honest and open with the community.
Such developers in other projects can be counted on one hand.
You are on the right track, thank you for doing this work for us.

I personally have not encountered any problems, I only use spot trading and Perps

J4J :rocket:

3 Likes

Is it possible to trigger controlled volatility on Solana to stress test everything before volatility happens in real time? Could maybe use catdets to run thousands of transactions together to cause controlled volatility and test systems? Would help prepare for situations similar to this in the future.

3 Likes

not a bad idea, it would be cool to see a test. I mean, we did test the LFG platform twice before it was ready

3 Likes

Thanks for being open with discourse, stuff breaks, appreciate all your hard work!

2 Likes