@maxkasy I think that the layout and flow of ideas are well done, it is already easy to follow even for a casual reader. In the finished product, having examples of the formulae being used in real life situations relevant to the field of AI will help drive the points home immensely.

I am in agreement on the level of principle both in regards to how goals should be conceptualized in terms of human welfare, and in how the means to propagate predictive algorythms (not to mention mundane social policy) should be democratized. However, (as much as I hate to admit it) I do not believe that our global society has the maturity to properly implement change in either of those areas as of yet, whether talking about policies surrounding AI or any social policy for that matter. In terms of social wellfare: In our globalized economy, it is impossible to divorce decisions made in one nation, to effects felt by others so formulae that seek to measure human wellfare would need to include all 8 billion of us and any change we would want to make would require a very robust and cooperative world government. Sadly this seems very far off, especially when considering the current regime's efforts to secure climate accords, and those deal with a literal existential threat! In addition to lack of consideration between nations, the governments within most (all?) these nations seem to be composed almost exclusively of members of very specific castes (I am using this term somewhat facetiously) and these actors seem to have very little incentive to even include members of lower social status in their wellfare calculations, acting as they are with predictable self interest. But how are we to blame them for this self interest, if it is displayed more or less equally by all members of their respective societies?

I do not mean to disuade your endeavors, by any means! But the realist in me insists that if we are to have some positive developements in the areas you examine, there may need to be some underlying changes in the individuals who comprise our global society before this can happen.