From billh@ssec.wisc.edu Sat Feb 7 11:08:22 2015 Date: Sat, 7 Feb 2015 11:08:05 -0600 (CST) From: Bill Hibbard To: Benja Fallenstein Cc: "researchassociates@intelligence.org" Subject: Re: Weekly report 2015-01-19 through 2015-01-25 Hi Benja, I enjoyed seeing you, Nate and Kaj at AAAI-15. On Tue, 27 Jan 2015, Benja Fallenstein wrote: > Also, I got to chat again with Bill Hibbard about his > idea for -- translating into my own way of thinking > about it -- using a bounded resource variant of AIXI > for some types of logical uncertainty tasks. Roughly, > the idea is illustrated by the following example: > Suppose that every time you upgrade the computing > hardware you run on, your average score increases > significantly. A probabilistic environment model > which is far too computationally constrained to > simulate the real world (which contains the real you) > might still be able to make probabilistic predictions > about your score if you buy additional hardware that > reflect this relationship. Bill had a version of this > system on his poster, and thinking about that and > talking to him about it helped me understand it > somewhat better than I had before. The self-modeling agent framework proposes to address several problems. As you note, one is predicting the value of increases to agent resources. Another is "inconsistency between the agent's utility function and its definition." One case of this problem is what Armstrong calls "motivated value selection" and describes as "a conflict between agents learning their future values and following their current values." In the self-modeling framework the agent learns a function from interaction history and proposed action to value. All elements of the agent defintion can be included in the function to be learned, including evolution of the agent's environment model and utility function as interaction history length increases. Thus this framework can avoid any "inconsistency between the agent's utility function and its definition." As defined in Section 8.4 of my book (http://arxiv.org/abs/1411.1373), the self- modeling framework proposes to address the problem of being predicted by other agents in the environment by learning when to choose actions stochastically. Best wishes, Bill