It seems obvious that one ought not be held responsible for things over which I have never had any control over. This thought needs to be refined, however, as it at first seems like the premise underlying libertarian free will.
I am constrained by certain factors external to myself. These include some factors of brain chemistry and bodily structure. My position is that only agential factors--for example, my intentions, desires, worldview--are relevant to responsibility. Our desires, etc., are subject to moral judgment because they are rightly part of our agential makeup.
The following is mostly a first attempt to deal with this--possibly a second attempt, but from a different direction from before. It is subject to revision in future posts or elsewhere.
What is it for something to be "rightly part of our agential makeup"? First, they must be the right kinds of things, that is, they must be mental. Second, they must be more or less well integrated into our whole agential makeup. Third, they must be basically stable.
The remainder of this post will serve to briefly elaborate and support these three criteria.
MentalsOnly mentals may be agential in a morally relevant way.
I mean to be quite broad here. By "mentals" I mean whatever mental phenomena are, whether properties or things. The criteria excludes mere objects and merely physical properties. Non-subjective stuff is not subject to moral critique, except within the context of subjective relations. I think this criteria is obvious, but it is also important, since it keeps this account from being, at least flatly, materialistic.
IntegratedOnly what is more or less well integrated into our whole agential makeup may be subject to moral critique.
This excludes phobias and other mental phenomena which neither support other aspects of our agentiality nor are so supported. A mental is supported by another agential feature when that agential feature provides a basis for the mental. Likewise, a mental supports another mental when it provides a basis for it. 'A is afraid of x because of agential feature y' is a case where y supports x, for example. This support only counts when the agent grants it, however. A must hold that y gives reason for the fear of x. That is, the support must be subjectively granted, it must exist as support within the agential makeup of the agent.
The question may arise as to how much integration there must be, and I doubt that answer can be clearly answered. Particularly given how non-differentiable mentals tend to be (that is, how hard it is to enumerate thoughts as distinct and fundamentally separate), it is hard to say how many links a given mental needs. It does seem like we can generally judge pretty well when something is anomalous in a person, however. The basic question is whether it fits into an outlook on the world which is perceived by the agent to be more or less coherent.
Some cases of irrationality may end up sneaking in, of course, but that seems like a point in favor of this theory, cf. Woody Allen. In these, the irrationality is seen as supporting aspects of the agential structure.
StabilityOnly features whose form can be traced more or less clearly through time may be subject to moral critique.
This may be considered as the diachronous version of integration. That is, where the above criteria holds that a mental must be supported or support, this criteria requires that a mental must have an origin which is not seen as an improper origin by the agent. Basically, this means that the agent must be able to maintain herself as a coherently storied being, and thus must be able to see herself as living out of or into a story.
Both this and the prior criteria have "more or less" in them, which permits for responsibility to admit of degrees.
Given these three criteria, agency may be distinguished from the ongoing flux of cause. I stand out by virtue of my being an integrated composition of mentals in an agential structure which develops organically through time.