Thanks for sharing a simple setup that avoids SwiftPM. It is nice to have more options.
Swift has to shed the perception that it only works on apple platforms. I've found the the C++ interop to be pretty good for my computer vision use cases.
The build process requires you to have a C library built with its own dependencies. I don’t think it’s very common to do that with SwiftPM, hence it would require extra work for it to happen.
CMake support started as a community project but was adopted officially by Apple after they started adopting some Swift in the compiler codebase – an extremely large and complex CMake project – so the support is extremely robust now.
It’s nice to see this addition. I’m not sure if Godot would be better off bridging OpenXR to apple’s ar compositer or do as these PRs implement.
It isn’t much work to bridge from a metal renderer to the ar compositor. There are nice, if under documented c apis for Compositor Services in visionOS. I don’t think this will end up being a heavy maintenance burden, but they should donate some headsets as the second vertex amplification doesn’t run in the simulator. The max threads per thread group also differ. So real hardware is needed to measure performance.
profiting 180 billion USD per year should put them in a position to also provide grants for hiring workforce for the necessary amount of years that will take to popularize their current and new AR/VR technology (if it'll ever be popular)
even if Godot starts also being the backend for non-gaming applications (which i don't know how the discussion about this went/is going), which AVP could also benefit, i doubt the investement of time to maintain this PR will pay itself, i.e. the few developers releasing software for Apple VR will make enough money and will donate enough for covering maintainer(s) to keep up with a (so far niche) new OS in Godot!
I built an interactive call graph interface targeting visionOS that I'll be open sourcing soon. My approach is to use the Language Server Protocol(LSP) as the itermediate format as many LSP implementations support the call hierarchy method. You can then proxy the commands to the headset for rendering. Using the LSP is nice because you can integrate the controls into small editor plugins and get live code analysis.
Most code analysis programs fail in some combination of editor integration, language support, iteration speed, or interactivity.
One of the big issues with visionOS right now is the divide between a full immersive space and a volume. There is a dev option to enable the virtual display to work in an immersive space, but normally a full metal rendered scene will hide your mac monitor. The volume requires RealityKit and provides no hand tracking data. My approach is to run the graph fruchterman-reingold spring embedder in a metal compute kernel updating a LowLevelMesh in a volume. The biggest limit I've faced is around 1000 input targets (graph nodes) is the most the RealitKyt renderer can support before the frame rate dips under 90Hz.
Good luck if you attempt visionOS, it is nearly undocumented.
Your implementation sounds great, and I'm looking forward to seeing it. Yes, the SDK is extremely obtuse and really not that easy to play with. I kinda was stubborn and refused to pick up the new rendering tool after just having learned SceneKit and then Metal, so I'm sure it's gotten at least a little easier, but the tools that Metal exposes just don't have analogues from what I can tell.
Keep me on your short list for beta users when the time comes =)
I thought Unity's Polyspatial was supposed to make this easier but definitely interested to see what you come up with..... AVP is a but underutilized by apps.
The cloth sim is GPU. It’s not that heavy currently, around 40k quads. You could go much heavier, easily 500k, but it also changed the cloth behaviour.
The limiting factor is actually not the sim, but the initial load time for the procedural geometry. I might try adding some subdivisions though..
It is mostly RAG, although I suppose that doesn't say much about the system: one thing I've found is that the way you clean and process the data substantially changes the quality of the results. I'll write a little blog post sharing some of the learnings!
If you feel up for it, you should share your email in the righthand "Unhappy with your results?" widget. My plan is to manually look into the disappointing searches and follow-up with better results for folks, in addition to fixing whatever can be fixed.
Agreed re: searching comments (which it indeed currently doesn't do).
i am not surprised that 'the way you clean and process the data substantially changes the quality of the results.'
can you share anything about your approach here?
I'll write up a little blog post once the traffic dies down a bit!
In the meantime, one thing that comes to mind is that simply embedding the whole contents of the webpages after scraping them didn't yield very good search results. As an example, an article about Python might only mention Python by name once. I found that trimming extraneous strings (e.g. menus, share links), and then extracting key themes + embedding those directly yielded much, much better results.
In our RAG pipeline we found that implementing HyDE also made a huge difference, maybe generating and embedding hypothetical user search queries (per document) would help here
Given a query, HyDE first zero-shot instructs an instruction-following language model (e.g. InstructGPT) to generate a hypothetical document. The document captures relevance patterns but is unreal and may contain false details. Then, an unsupervised contrastively learned encoder~(e.g. Contriever) encodes the document into an embedding vector. This vector identifies a neighborhood in the corpus embedding space, where similar real documents are retrieved based on vector similarity. This second step ground the generated document to the actual corpus, with the encoder's dense bottleneck filtering out the incorrect details.
Looks promising. It would be nice if the XR components exposed more configuration options. For example I don't see a way to use the transient pointer of the apple headset nor the depth buffer of the Quest 3.
Hi! @threlte/XR maintainer here. Both of those features should be pretty easy to use with Threlte since the package exposes the underlying WebXR APIs. Here's a repl example with both transient pointers and depth sensing:
There's likely an opportunity to do some abstraction here since it's a little clunky, but I've been holding off for the moment since these features are pretty new. We could potentially update the docs to make it clearer that this kind of configuring is possible. If you have any suggestions for either abstractions that we could add or more clear docs then don't hesitate, I'm all ears!
Amazon Neptune supported both Gremlin and Cypher. AWS states in a blog post[0] that Cypher is the pathway to GQL. That aligns with the blog post. Personally I was hoping Gremlin would go farther, but it appears GQL is on track to be the unifying language for property graphs in a few years.
Guess it is time to learn Cypher as increasingly graph db vendors are adopting GQL. Is there an existing GraphDB that implements the GQL spec?
More open source projects should have fun text and drawings in them like the Tinkerpop[1].
Maybe all these years of staring into the uncaring blackness of my terminal has damaged my brain, but there are some real gems in "Tales from the TinkerPop."
"Grasily had been stationed at vertex v[32] for as long as he could remember. ... Grasily's commanding officer was org.apache.tinkerpop.gremlin.process.traversal.Traverser@76a36b71"
"The famous mystic traverser Gri Gurobindo (ग्री गुरोबिंदो) once said:
My single step in the graph is composed of the steps of numerous other traversers."
Our computers are at least 10,000 times faster than decades ago. Yet, are we having 10,000 times the fun?
It's neat, but I have a feeling this will result in very large downloads and long load times. Like the Unity WebGL exports of the past.
The car example https://car.tiwsamples.com/ took 120 seconds to load, making 1820 network requests, and transferred 392MB of data / 487MB decompressed.
You can deliver a significantly better user experience by building for the web directly.
There are definitely pros and cons, just like there is for using web-first 3D frameworks and engines. In terms of Unreal, asset streaming is a must and helps significantly here. Also, these demos are going to continue to be optimized for file size/load time, so you can expect speedups over what we've currently made available.
This represents a way for Unreal developers to export their authored content to the web. It is an option, but is by no means the only way to do 3D in the browser. What web native frameworks lack is the rich functionality that comes with game engines such as Unity and Unreal.
There has been a chicken and egg problem for gpu intensive media on the web. WebGL overhead limits complexity -> demanding applications remain native -> tooling for WebGL/WebGPU is limited or lags behind.
I bet these are some of the most complex WebGPU programs.
Congrats on the launch! Good luck on the optimization.
Swift has to shed the perception that it only works on apple platforms. I've found the the C++ interop to be pretty good for my computer vision use cases.