The Luck Ran Out

This week we were expecting, and received, another slew of rolling restarts as part of the final push (or so Oz Linden had said) to get SL totally into the cloud.

The Rolls on Monday and Wednesday appeared to go to plan, but then it became clear that there was a problem updating EEP settings on all regions. It seemed that the second update to the servers had introduced either a bug or a regression and EEP function was corrupted.

The solution was a set of back-end and server-side changes necessitating a further roll to all sims on Friday.

All well and good except….. the rolls did not occur when they were supposed to though nearly all sims were rolled in due course. I say nearly all because my home region (and several others at the least since there must have been other regions on that server) did not get the updated software version and as a result EEP still could not be customised. However that was not all! The “back-end” changes had caused a communications glitch for scripts that needed to communicate with external servers. That issue was quickly and efficiently dealt with and the weekend began.

However , when I contacted SL support regarding the missed regions, I was astonished to find that the support operative had no idea what to do about it and eventually told me that I (and the other region owners, I assume) would have to wait for the next round of rolling restarts before my region could be put on the fixed server version! It not transpires that LL, at the weekend at least, cannot effect rolls and thus correct errors!

I was offered a region restart, but since that would not change the server version, it was irrelevant. So as of now I await developments, but I suspect this is not going to be performed any time very soon.

A shame, it had all been going so smoothly until this week and now the uplift, as far as I am concerned, has crashed and burned.

One aspect of this bothers me: just how much control does LL actually have on these uplifted AWS servers and are we now dependant upon AWS techs to perform rolls? That seens to be an unreasonable penalty to pay for the presumable saving in running costs.

ETA Jan12th 20201: Just to clarify all the above, I was told later that week by Mazidox Linden that indeed some regions had been deliberately NOT rolled to give them a comparison for the new code rolled to most regions, though there was no explanation WHY they decided to do such a thing. Subsequently it became clear to users that all was not quite as it seemed and LL were not inclined to explain their actions.

Over the period between then and now a few more restarts have occurred, the last one on Friday last between 10am and 2pm SLT (brilliant timing, one of the peak periods for events for European users) was simply to re-roll identical code but since the same code could not be rolled twice (why?) it was renumbered. No explanation has been forthcoming as to why such an action was needed and no performance issues seem to have come as a result, but while LL have trumpeted the end of the uplift, they tell us that some systems are still not working due to code chages being necessary. One casualty of this is the World Map, which still is non-functional to a large extent. This has made sailing long distances a greater lottery than ever, though I have been told that the Gulf of Jeogeot is now mapped and good for sailing!

EEP! A New Firestorm!

The new full-release EEP-equipped Firestorm Viewer was unveiled during the night (in my timezone) of 9-10th December. It contains all the usual upgrades and brings it pretty close to parity with the current release LL viewer. Oh by the way, EEP = Environment Enchancement Project…well it was a project once, about 2 years ago!

The BIG THING about this release was the EEP code which totally changes the way our environment is presented and altered. It is the biggest change since the introduction of materials to the Windlight rendering system, and frankly it is the biggest change to the way we “see” SL since the introduction of Windlight to SL some 12 years ago.

EEP is not universally liked by users and this release by Firestorm has not met with universal praise from its users. This is a shame because in its highly revised and improved form as it is in both FS and the “official” viewer, it is quite an advance.

Together with the substantial changes made to the render pathway ( nicknamed “Love Me Render” by LL…[why DO they do this, its cringeworthy]) your SL is going to look far better than it has done to date. For one thing, you are now allowed to use a great deal more texture memory if your GPU has it and the upshot of that is much finer detail on nearly all items and most noticeably materials equipped mesh clothing.

EEP skies and water are radically changed….the stars are back to their correct size and they twinkle! Day length and type is now customisable (though I haven’t played with that yet). Some folk are ranting about the loss of their favourite windlight settings and whether ot not this will be a significant issue remains to be seen.

Some older and more basic hardware will struggle with this new system just as older hardware struggled with Windlight back in 2008-9 (mine did and I had to upgrade my graphics card in the end to make it useable).

This is a “first look” and I will post more in due course but the transition so far has been relatively painless and the plus points are significant. There are the now expected bug fixes and feature refinements in this version and so far I have noticed no new bugs apart from a couple of odd rendering issues with non-projector lights (which are very old now) and flexiprim or alpha channel containing surfaces, which are also very old (a bit like me, I guess). Nothing is perfect but as a way into the realms of EEP, this is a good effort by the capable and hard-working Firestorm Development team.

An Uplifting Experience

The cuurent work of LL to “uplift” all SL functions to the Amazon Web Services cloud servers has raised so much dust that it has been hard to see where we were going. However as of this week my home on SL’s Main Server made it into the cloud and since nearly all the other server-related functions are already there I was hoping for a significant improvement in performance.

I was NOT disappointed. There was no change to client frame-rate, which was something a lot of folk seemed to be expecting (why I cannot understand since frame rate and rendering lag are entirely down to one’s own computer, its graphics processor and one’s connection speed to SL). What I am seeing so far is a slight reduction in ping and rock steady time dilation at 0.999. Object and texture rezzing seems much quicker and while some folk say that scripts are occupying more server time than was the case at the datacentre, I must be honest and say I am not seeing it.
Scripts are running better even though % scripts run per frame has not changed.

I was hearing of troublesome region border crossings on Blake Sea but I have not observed this in the past few days…maybe that BUG has been squashed! There are one of two new ones, as one might expect but none affects me in my day-to-day SL. Profiles (individual and group) load slowly at first but do load consistently, so I can live with that for now. Teleports for me have been fine, with only a few failures, scripted object property changes seem fine as do all editing funtions that I have used, so for now I am happy with the result of LL’s hard work.

Fingers crossed, this stage of the uplift seems to be going better than nearly everyone had expected and the whole operation should be completed well before Christmas. With all the restrictions placed upon our First Lives, it is nice to know that our Second Lives seem to be running well again!

ETA Sat Nov 21st:

Linden Lab completed the Uplift of SL’s simulators to the cloud this week, just two regions have been left on datacentre servers for debugging purposes (they are essentially sandboxes with quite short return times). The uplift appears so far to have gone without any further major hitches, the small number of bugs found are now to be tracked down and squashed, according to LL sources.

Just when you thought it was safe to…….

Go sailing. First trip NW of the Blake Sea we had a pleasant trip on my partner’s Catamaran, no real issues at sim crossings. Same evening, crossing into a populated but not overstuffed region (same area NW of Blake)…BOOM! At the crossing the boat vanished though we were still sat on something….skittered across one region out of control, then a second…tried to stand and neither of us could. Finally at the next border we were unceremoniously dumped on the seabed. Tried to TP out…SL said no. three tries later I gave up and logged out. Tried to relog…SL said no, the logging out process was still ongoing, please try in a few minutes. I did. Five times over the next 10 minutes. Clearly that region was not letting go of my avatar. So I changed my login to Home, tried again …SL said the region was STILL logging me out!!

Finally, after ten more attempts I got logged in at home. It took my partner a further ten minutes to get home the same way. LL Support had no idea why it would take so long but reminded me that using the Firestorm viewer was probably the reason. I was polite but begged to disagree.

So…after over a year and umpteen “improvements” to region hand offs we are still at square one….I shudder to think how it will go when the servers are virtualized in the cloud. And, guess what? We have both rather “gone off” sailing for now!

Edit to add on 22nd July:
From recent comments on the Technology Blog, folk who have bravely gone sailing on LL’s cloud-based recreation of Blake Sea on Aditi (the Beta grid) seem to be having very mixed experiences. One thing seems clear on the cloud there are no half-measures, either the crossing works well or the object, be it boat, plane or whatever vanishes and you end up unceremoniously dumped on the ocean floor. Apparently another crossing failure mode results in you banging your virual head on the region border as though a glass wall had been errected arount the region. The information I take away for this is that there is a lot more to do!

Further edit Sept 9th:
Following some extensive research on the BetaGrid Aditi connected to the much anticipated move of Server functions to the cloud, for once LL sought the involvement of residents and as a result of some very positive feedback, LL introduced some new region-crossing code to Agni…and it worked! Region crossings were vastly improved and at last sailing was not a pursuit that required large amounts of luck if you were to complete a race or even a pleasure trip! This continued until the last Roll, to Server Version 547626, which contained a fix for an outstanding Server script bug detailed in SL-13785. LL believed it addressed a significant issue when crossing between regions with scripted objects(craft) they proudly announced: “SL-13785 improves some instances where vehicles would stop responding to control inputs after a region crossing. We don’t believe we’ve fixed all instances of this issue yet, but this patch should offer some relief for the issue.”
Sadly it didn’t. In fact it almost completely ruined region crossings! Sufficiently so for some very pointed comment to be made on the SL Server Forum. Oh well, it was nice while it lasted! This week there is a hiatus due to Labor Day and the anniversary of 9/11. so we must wait and see if anything is rolled to the RC channels next week that mitigates this FUBAR.

Going Viral – The Value of SL

Sorry, that is probably in bad taste but that’s me. It is interesting to see the wholly expected upkick in online numbers for SL these last couple of days…more than 10% so far. When many of us worldwide are told in no uncertain terms to “Stay at Home”, most of us do. It is clear also that a small percentage of many poulations either think they are exempt from such restrictions or (more likely in my opinion) just don’t give a monkey’s.
Over the years I have defended the facilities of SecondLife to critics of many types…now perhaps, some will see the value of our online social world. SL is not twitter, it is not WhatsApp, it is NOT Facebook, it is something altogether more immediate and intimate. For some of us who, thanks to various forms of physical disfunction, have come to rely on SL for our interractions and social life, that folk are now realising the value of SL comes as an “oh at last you get the point” moment.
In SL we must never forget that our social world (no it is NOT a game, games are competitive and have defined goals) is maintained and operated by RL folk who, we must hope, can keep the lights on while remaining safe themselves.
I look forward to seeing some of those who deliberately or inadvertently read this inworld soon.

Deja Vu?

Just recently, and despite my not having taken any formal part in SL Ponyplay for several years now, I have been following a fierce row over the use of Alt accounts in pony polo. That there is a schism in SL pony polo anyway is plain for all to see with adherents to the “old” classic style of field at odds with those adopting the “new” sloped field (the logic behind which I still fail to see after reading many articles on the subject).
But recently a strange row blew up and in the midst of all the pompous invective, the identity of some of the “alts” was publically announced. Now that, as I understand it is a slam-dunk ToS infringement. However negatively one might feel about such accounts it is not permitted to “out” such folk. It is clumsy at best and seriously offensive at worst. I wrote an article about Ponyplay etiquette some time ago, but it bears re-reading in the light of recent developments:

https://ayesha001.wordpress.com/2017/02/23/equine-etiquette-in-sl-an-outmoded-concept/

Maybe I AM old-fashioned, but what I said there is every bit as relevant here. It is long past time that SL Ponyplay drew back from this confrontational drama. If it does not, it will wither yet further.

Inventory Issue

Last weekend saw SL hit by a vicious outage to the inventory servers, unattributable, for once, to the Lab themselves. On Friday many folk found themselves unable to remove attachments or rez objects from about 1pm SLT, at that point logins seemed to be affected too and few were able to log in.
It took a good hour before LL officially recognised the issue in a post on the Grid Status page and a further hour before they reported the issue at least temporarily fixed.

The next evening the issue returned with a vengeance and this time the issue resisted being fixed for many hours, resulting in inworld numbers plummeting to under 20K (at which point I must wonder just WHAT this buffer of 15-20K is). I was one of many unable to login for over 12 hours.

Linden Lab published an explanation of the outage in a Blog post a couple of days later: https://community.secondlife.com/blogs/entry/2901-the-sunday-that-was-not-a-fun-day/. It transpired that connectivity issues within SL’s own network but not run by themselves was at fault. Now for many that was an end to the issue but for some it was not. Subsequent to this outage many folk, using all types of viewers, found their inventory either corrupted or missing to a variable extent. Now LL are slowly correcting (or attempting to correct) these errors, since they are demonstrably NOT viewer cache issues. I know of a friend who is missing some 20K items from her inventory and this is identical on both Firestorm and on the LL viewer. We must wait and see how LL deal with this issue. My friend has raised a ticket on the matter and must await LL’s response.

Script Woes

For the past few weeks we in SL have been suffering various script communication issues. Ironically the root cause of many of these woes is LL’s attempts to improve script scheduling and thus increase the percentage of scripts that can run in each frame of a simulation (ideal running is 45 FPS).
It would be silly to BLAME LL for these issues as it was a genuine attempt on their part to improve matters, though just what changed in the first place to cause the drop in script run no-one seems to know. We have been bemoaning the apparent drop in script run over the past two years now and various abortive attempts to improve affairs have been undertaken; net result no change.
Rider Linden and others have mounted a recent and more aggressive attempt at solving this issue and unfortunately the results were equally aggressive …and negative! Now no matter where in SL one is, the likelihood of a script resetting to its default state has become significantly higher, coupled with a marginal, and I DO mean marginal (c5%) increase in overall script-run performance.
The changes have been to script-scheduling on the server/simulator and the scripts are now prone to miscommunication, most obvious to those running remote scripted attachments (usually in RLV(a)) when moving between regions. The worst of this occurred a little over two weeks ago and thankfully the code responsible was withdrawn by LL since there were additional most undesirable side effects. Some folk were a little disappointed with the “roll-back”, since it had been accompanied by a noticeable increase in script run performance (c25%); that this was obtained at the cost of certain scripts no longer communicating AT ALL did not, apparently, bother them!
Now Linden Lab’s finest coders are trying again to effect a rescheduling of script events and just what the resulting effects will be we must wait and see! ‘ee innit fun!

Firestorm 6.3.2 Bo(o)M

Well, Firestorm’s Bakes on Mesh viewer is here. There are many other updates, tweaks and minor changes since the last one, not least some further upgrades to the mesh-rendering process, gleaned from LL’s latest version of the “Love Me Render” code. It’s worth pointing out that the viewer has been available for well over a week now so this is a comment after a fair spell of use.

Now personally I can’t get excited about BoM, I don’t use tattoos and the constraints of not having specular- or bump-mapping on baked textures seem a retrograde step to me. Others may well differ in their opinions, but until I am using the updated Maitreya body (coming “soon”) I am holding my opinion.
All I will say is that this update seems to render mesh bodies faster than the last viewer, which itself was a BIG improvement. I have been able to lower my routine object LOD now to 3.0 and to 3.25 in places with a lot of complex sculpts. Up until now I have needed 3.75 or thereabouts to achieve satisfactory rendering, with all the attendant load on my GPU and resultant lower frame rates, so that is a win in my book. Other details are to be found in Inara Pey’s more in depth assessment: https://modemworld.me/2019/10/01/firestorm-6-3-2-welcome-to-bakes-on-mesh/.

It’s important to mention that Inara’s blog routinely gives an insight into current LL programmes and projects and if you want to get a handle on how LL is actually working to improve our Second Lives, I heartily recommend her Blog.

Firestorm 6.2.4

There is a new version of Firestorm available now. This version is not a quantum-leap in any particular direction and is unlikely to sway users of other viewers to the cause, were it not for the woeful issues that have hamstrung Singularity users recently. There is a fix for an issue afflicting users of Intel integrated graphics as well.

The update is more maintenance than anything else apart from some significant changes to Estate Management tools (EAM) and the incorporation of the changes to render pathway introduced by LL in their “Love Me Render” RC viewer. (Why do they do that, their sense of humour is worse than mine, if that is possible). The changes are not immediately obvious to me after several hours of use, apart from that it seems mesh attachments go immediately to where they ought to be attached, rather than orbiting the avatar for 30 seconds or so, after entering a parcel/region. I assume this indicates better sim to client communication.

The long-awaited Bakes on Mesh seem to be no nearer and EEP bugs are still to be cured so LL are not issuing these for mainstream operations yet. Vision-spheres are still not incorporated and I rather get the impression from the accompanying changelog notes that Kitty Barnett is becoming short-tempered about the issue, pointing out yet again that her interpretation of vision-spheres is done but is awaiting some greater project’s completion. Frankly I have stopped waiting for that. If you MUST have vision spheres, the best solution would be to use either Marine Kelley’s RLV or the Kokua TPV. If there are other solutions I do not know of them.

All in all FS 6.2.4 is a competent update and I found it easy to install despite the certificate issue causing my Firewall to flag up doubts about the source. As ever use the official FS website to download.