Apple controls the app, the network, and the devices that gather the location information.
It seems like in order to show me where my missing item is, they have to be able to determine my device’s information. They might claim to choose not to do that correlation until I open the app and go looking, but once I do, it seems obvious that Apple can tell where my device is (in order that they can tell me where it is in their Find My app).
The "device information" is likely an opaque ID, and the "location" will likely be encrypted using an asymmetric key-pair set up during the pairing process, so the only thing capable of decrypting the location will be your phone.
The part you're correct about is that as they control the device, there's nothing saying they can't build a backdoor into it that reports the information (ie. location) back to them once it hits your phone. And we're also taking it on trust that it works the way they say it works, as it's not open-source.
But as someone else commented, eventually you have to trust something.
The realistic threat likely isn’t a designed backdoor, but some late-stage bug (especially if server-side) that caused part of the E2E encryption or privacy story to get punted. Who would really block ship on that, esp. with hardware impact?
The way it works is that the location is encrypted with a key that is only on your device, same as iMessage. So they can't directly decrypt that info.
It's possible in theory they could add hooks into the OS to then do that next time you use the key (e.g. effectively a baked in backdoor) or maybe in some cases the key can be extracted from an iCloud backup or similar (I'm not 100% sure how those keys are stored, but it's likely detailed in their security documentation) but in general the service itself cannot see the encrypted device location.
This is of course true but at some point you have to pick your battles. There is no FOSS product which does this and there probably never will be. And it’s not in Apple’s business model to try to collect this data.
It seems like in order to show me where my missing item is, they have to be able to determine my device’s information. They might claim to choose not to do that correlation until I open the app and go looking, but once I do, it seems obvious that Apple can tell where my device is (in order that they can tell me where it is in their Find My app).