{
"summary": "RemoteActorStore fetchRemoteActor",
"mediaType": "text/html",
"@context": [
"https://www.w3.org/ns/activitystreams",
"https://forgefed.org/ns"
],
"context": "https://vervis.peers.community/decks/br6Go",
"history": "https://vervis.peers.community/decks/br6Go/tickets/r7dDo/events",
"published": "2023-04-03T07:07:55.69025Z",
"dependants": "https://vervis.peers.community/decks/br6Go/tickets/r7dDo/rdeps",
"dependencies": "https://vervis.peers.community/decks/br6Go/tickets/r7dDo/deps",
"content": "<pre><code> -- Without concurrency in mind, what do we want to do:\n --\n -- * check if a URA exists\n -- * if not:\n -- - insert new RA\n -- - done\n -- * if yes:\n -- - get all the URA's unlinked deliveries\n -- - delete them from DB\n -- - insert new RA\n -- - insert linked deliveries for them\n --\n -- but with concurrency present, we can easily ruin things.\n --\n -- The goal is that when the periodic delivery runs, and\n -- reads all the unlinked deliveries not already running,\n -- it doesn't need to check whether their URA since got a\n -- matching RA. It doesn't need to, because whenever a new\n -- RA gets inserted, all unlinked deliveries switch to\n -- linked, and probably the URA gets removed. So, we don't\n -- need to worry about it.\n --\n -- Is this requirement even possible? I like to think that\n -- it is: There's exactly 1 place now where actors get\n -- fetched, and that place is right here. So we have a\n -- chance to maintain such conditions.\n --\n -- So, once we've fetched our nice little actor, which\n -- things may be happening concurrently?\n --\n -- * In outbox post handlers, unlinked deliveries may be\n -- running, waiting for us to give them the new RA\n -- * In the periodic postman, unlinked deliveries many be\n -- running, waiting for us to give them the new RA\n -- * New unlinked deliveries may get inserted. If the URA\n -- has 'since' set, outbox post handlers may skip\n -- trying to fetch the RA, and instead just proceed to\n -- inserting UnlinkedDeliveries.\n -- * Hmmm UnlinkedDeliveries might get deleted? Because\n -- the URA has been unreachable for too long.\n -- * We might have unlinked deliveries in running state,\n -- but which aren't waiting for us. For example, they\n -- got inserted to the DB, but haven't reached the\n -- point in the code, where they readMVar.\n --\n -- Once we commit the RA insertion, there will be no new\n -- uses for the URA. We may need to adapt some code to\n -- allow for RA and URA to coexist though, unless we can\n -- guarantee here that it's never the case.\n --\n -- Persistent's 'delete' does nothing for nonexistent\n -- records, so we can safely delete things here even if\n -- they'll get deleted again while running deliveries.\n --\n -- So, again, the situation is that if a URA exists, we\n -- want to:\n --\n -- - get all the URA's unlinked deliveries\n -- - delete them from DB\n -- - insert new RA\n -- - insert linked deliveries for them\n --\n -- The unlinked deliveries that currently exist in the\n -- database may be:\n --\n -- * Just got inserted by outbox post, and done with\n -- * Just got inserted by outbox post, and in a bit\n -- they'll be waiting for us\n -- * Got inserted by outbox post and waiting for us\n -- * Resting in DB, waiting for periodic run\n -- * Got picked by periodic run, some about to be deleted\n -- due to unreachability and some will do\n -- fetchRemoteActor and eventually wait for us\n --\n -- All the deliveries that are going to look for an RA, we\n -- don't have to worry about them. Because whether they\n -- wait for us or grab the RA from the DB after we finish,\n -- they'll switch to a linked Delivery and stop using the\n -- URA. I think.\n --\n -- Irony: This very piece of code may be running because\n -- some unlinked deliveries want the RA. So, both them and\n -- us and going to worry and switching from UDL to DL?\n --\n -- In outbox post, they'd be delivering new UDLs where the\n -- URA is either new, or it's "since" is Nothing. But when\n -- they run us, why not switch *all* the UDLs for that URA\n -- to DLs?\n --\n -- What if we decide, that we do all the UDL-to-DL\n -- switching here?\n --\n -- Let's check the outbox post code, would it be okay?\n --\n -- The behavior after fetchRemoteActor, if we ignore the\n -- try-one-per-host optimization, is:\n --\n -- * If it fails, update the URA's "since" to Just now,\n -- and UDL's "running" to False\n -- * If successful, try to deliver to inbox:\n -- - If fails: update RA's "since" to Just now,\n -- delete the UDL and insert a DL\n -- - If success: delete the UDL\n --\n -- So what we could do in fetchRemoteActor is:\n --\n -- * If fetching fails, update URA "since" to Just now,\n -- if it was previously set to Nothing\n -- * If fetching succeeds, ummmm this is a problem. We\n -- Right now, a UDL stays existing until after delivery\n -- to inbox. This means, we can't delete UDLs right\n -- after fetching RA. Alternatively, we could decide\n -- that upon RA fetch, we instantly delete the UDL and\n -- insert a new DL, and only then, we try to deliver to\n -- inbox. So, we delete all the UDLs here? But that\n -- means we must insert new DLs for them, right? We\n -- can't just delete. The code that waits for the RA\n -- though, what is it supposed to do? I guess instead\n -- of inserting a DL, it will require to *fetch it from\n -- the DB*.\n --\n -- So, summary of changes:\n --\n -- * If fetching fails, update URA since\n -- * If fetching succeeds, delete all URA's UDLs, delete\n -- URA, insert RA, insert DLs</code></pre>",
"replies": "https://vervis.peers.community/decks/br6Go/tickets/r7dDo/discussion",
"isResolved": false,
"followers": "https://vervis.peers.community/decks/br6Go/tickets/r7dDo/followers",
"source": {
"mediaType": "text/markdown; variant=Pandoc",
"content": "```\n -- Without concurrency in mind, what do we want to do:\n --\n -- * check if a URA exists\n -- * if not:\n -- - insert new RA\n -- - done\n -- * if yes:\n -- - get all the URA's unlinked deliveries\n -- - delete them from DB\n -- - insert new RA\n -- - insert linked deliveries for them\n --\n -- but with concurrency present, we can easily ruin things.\n --\n -- The goal is that when the periodic delivery runs, and\n -- reads all the unlinked deliveries not already running,\n -- it doesn't need to check whether their URA since got a\n -- matching RA. It doesn't need to, because whenever a new\n -- RA gets inserted, all unlinked deliveries switch to\n -- linked, and probably the URA gets removed. So, we don't\n -- need to worry about it.\n --\n -- Is this requirement even possible? I like to think that\n -- it is: There's exactly 1 place now where actors get\n -- fetched, and that place is right here. So we have a\n -- chance to maintain such conditions.\n --\n -- So, once we've fetched our nice little actor, which\n -- things may be happening concurrently?\n --\n -- * In outbox post handlers, unlinked deliveries may be\n -- running, waiting for us to give them the new RA\n -- * In the periodic postman, unlinked deliveries many be\n -- running, waiting for us to give them the new RA\n -- * New unlinked deliveries may get inserted. If the URA\n -- has 'since' set, outbox post handlers may skip\n -- trying to fetch the RA, and instead just proceed to\n -- inserting UnlinkedDeliveries.\n -- * Hmmm UnlinkedDeliveries might get deleted? Because\n -- the URA has been unreachable for too long.\n -- * We might have unlinked deliveries in running state,\n -- but which aren't waiting for us. For example, they\n -- got inserted to the DB, but haven't reached the\n -- point in the code, where they readMVar.\n --\n -- Once we commit the RA insertion, there will be no new\n -- uses for the URA. We may need to adapt some code to\n -- allow for RA and URA to coexist though, unless we can\n -- guarantee here that it's never the case.\n --\n -- Persistent's 'delete' does nothing for nonexistent\n -- records, so we can safely delete things here even if\n -- they'll get deleted again while running deliveries.\n --\n -- So, again, the situation is that if a URA exists, we\n -- want to:\n --\n -- - get all the URA's unlinked deliveries\n -- - delete them from DB\n -- - insert new RA\n -- - insert linked deliveries for them\n --\n -- The unlinked deliveries that currently exist in the\n -- database may be:\n --\n -- * Just got inserted by outbox post, and done with\n -- * Just got inserted by outbox post, and in a bit\n -- they'll be waiting for us\n -- * Got inserted by outbox post and waiting for us\n -- * Resting in DB, waiting for periodic run\n -- * Got picked by periodic run, some about to be deleted\n -- due to unreachability and some will do\n -- fetchRemoteActor and eventually wait for us\n --\n -- All the deliveries that are going to look for an RA, we\n -- don't have to worry about them. Because whether they\n -- wait for us or grab the RA from the DB after we finish,\n -- they'll switch to a linked Delivery and stop using the\n -- URA. I think.\n --\n -- Irony: This very piece of code may be running because\n -- some unlinked deliveries want the RA. So, both them and\n -- us and going to worry and switching from UDL to DL?\n --\n -- In outbox post, they'd be delivering new UDLs where the\n -- URA is either new, or it's \"since\" is Nothing. But when\n -- they run us, why not switch *all* the UDLs for that URA\n -- to DLs?\n --\n -- What if we decide, that we do all the UDL-to-DL\n -- switching here?\n --\n -- Let's check the outbox post code, would it be okay?\n --\n -- The behavior after fetchRemoteActor, if we ignore the\n -- try-one-per-host optimization, is:\n --\n -- * If it fails, update the URA's \"since\" to Just now,\n -- and UDL's \"running\" to False\n -- * If successful, try to deliver to inbox:\n -- - If fails: update RA's \"since\" to Just now,\n -- delete the UDL and insert a DL\n -- - If success: delete the UDL\n --\n -- So what we could do in fetchRemoteActor is:\n --\n -- * If fetching fails, update URA \"since\" to Just now,\n -- if it was previously set to Nothing\n -- * If fetching succeeds, ummmm this is a problem. We\n -- Right now, a UDL stays existing until after delivery\n -- to inbox. This means, we can't delete UDLs right\n -- after fetching RA. Alternatively, we could decide\n -- that upon RA fetch, we instantly delete the UDL and\n -- insert a new DL, and only then, we try to deliver to\n -- inbox. So, we delete all the UDLs here? But that\n -- means we must insert new DLs for them, right? We\n -- can't just delete. The code that waits for the RA\n -- though, what is it supposed to do? I guess instead\n -- of inserting a DL, it will require to *fetch it from\n -- the DB*.\n --\n -- So, summary of changes:\n --\n -- * If fetching fails, update URA since\n -- * If fetching succeeds, delete all URA's UDLs, delete\n -- URA, insert RA, insert DLs\n```"
},
"id": "https://vervis.peers.community/decks/br6Go/tickets/r7dDo",
"type": "Ticket",
"attributedTo": "https://vervis.peers.community/people/VvM9v"
}