![]() ![]() Some mechanisms that lead to tail impact include multiplicative processes (judging and selecting researchers and groups of people based on a variety of factors), preferential attachment (investing in doing well in one’s research career early on), and the “edge of chaos” heuristic (transforming a small piece of a chaotic area into something ordered). ![]() Research should have a tractable tail impact and avoid producing capabilities externalities. With diversification, and because of the high uncertainty of AGI research, we allow ourselves not to give conclusive answers to difficult/uncertain questions and instead optimize for doable research tasks that decrease x-risk. All these lead to thinking that it’ll be best to diversify our research priorities. Consequently, proposals to first align smaller models and then scale them up to bigger ones are misleading. It’s worth noting that we should not expect larger systems to behave like smaller systems because scaling brings new qualitative features to the surface. Current research agendas treat intelligent systems as mathematical objects while it makes more sense to represent them in terms of complex systems. Moreover, the crucial aspects of the system are discovered by accident meaning that there are no explicit rules to follow in the pursuit of scientific discovery. This emphasizes the urgent need for more empirical work. Importantly, the thorough study of a complex system does not predict its failure mode. The usefulness of this approach relies on the predictive power of complex system models. Furthermore, the research community can be viewed as a complex system as well, just like the organizations that work in the area. One important observation is that deep learning has many complex system-type features. It is then critical to detect the contributing factors of the highest value. There are many reasons why safety has been neglected so far, and being attentive to them will increase our chances of success. Having well-defined problems also helps avoid trying to convince technopositive ML researchers that their work might be extremely dangerous. Thus, we should prioritize making research agendas as clear and interesting as possible so that such researchers are incentivized to pursue them. So far, it hasn’t been easy to attract top researchers based solely on pecuniary incentives: these people are internally motivated. Who gets to choose AI safety research directions is another key contributing factor. The criticisms against AI safety which come both from AI ethics (bias, equality, etc.) and the broader discourse, call for strengthening the safety community and increasing its reliability. Safety won’t become the community norm immediately it’s necessary to have a good understanding of what safety entails as well as develop the infrastructure for AI safety research. It’s crucial to contextualize the AGI x-risk by developing a safety culture. Moreover, how people think about “tail risks” i.e., rare risks is currently a problem in dealing with AI x-risk. Bettering people’s epistemics should generally make them better Bayesian thinkers with the obvious benefits that follow. Analyzing the AI x-risk at the societal level is also fruitful. For AI safety, forecasting and rationality have an evident positive effect that is difficult to measure as “increasing the intelligence of the system” (where the system is the safety community). ![]() The accurate description of systemic factors makes the value of their effect clearer even when such an effect doesn’t point to a specific measurable outcome, e.g., trying a new set of experiments. This means that we should be mindful of the value of contributing factors that aren’t, strictly speaking, “direct impact”, i.e., researchers working on the technical/mathematical/engineering safety solutions. The first insight is that if we aim at solving alignment, we should have a broader and more inclusive definition of impact. The insights from complex systems study help reframe the problem of how to make AI safer with consideration for the multiple dimensions of the problem. As a result, traditional methods of reductionism and statistical analysis respectively are inadequate. To examine AI safety as a complex system means to understand that various aspects of the problem are too interconnected to effectively be broken down into smaller parts and too organized from the perspective of statistics. This post is my summary of “Pragmatic AI Safety” on complex systems and capabilities externalities. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |