1 00:00:00,000 --> 00:00:03,801 welcome to another episode of the Democracy Innovator Podcast and our guest 2 00:00:03,801 --> 00:00:06,803 of today is Alberto Fernandez Hibaja. 3 00:00:06,724 --> 00:00:07,724 Thank you, Alessandro. 4 00:00:06,803 --> 00:00:10,825 And ~ so welcome to the podcast and thank you for your time. 5 00:00:10,825 --> 00:00:16,838 And and as f I mean the first question 6 00:00:16,838 --> 00:00:20,129 is if you'd like to tell us something about you, 7 00:00:20,129 --> 00:00:23,111 something about your background or personal life 8 00:00:20,580 --> 00:00:24,852 Sure. yeah, my name is Alberto Fernandez Ghibaja. 9 00:00:24,852 --> 00:00:28,254 I'm currently leading International IDEA 10 00:00:28,254 --> 00:00:34,378 ~ global work on everything at the intersection 11 00:00:34,378 --> 00:00:38,890 of technology and democracy. For those of you I'm familiar with International IDEA, 12 00:00:38,890 --> 00:00:43,723 we are an intergovernmental organization with 35 member states ~ 13 00:00:43,723 --> 00:00:47,745 and global presence and member states from all over the world, 14 00:00:47,745 --> 00:00:48,806 all of them democracies. 15 00:00:49,560 --> 00:00:53,879 with just one mandate, which is to support democracy worldwide. 16 00:01:02,045 --> 00:01:08,128 Thank you. And and ~ how did you come to this 17 00:01:08,128 --> 00:01:13,320 h when you realized that technology could be interesting for democracy? 18 00:01:10,394 --> 00:01:11,394 That's 19 00:01:10,675 --> 00:01:16,800 an interesting question I am of I'm old enough to be of those that generation that 20 00:01:16,800 --> 00:01:22,305 we started working on technology as From from a positive point 21 00:01:22,305 --> 00:01:27,730 of view we thought and we joined the wave 22 00:01:28,111 --> 00:01:32,635 of ~ enthusiasm that at the beginning of 23 00:01:32,635 --> 00:01:35,586 the of the 2010 24 00:01:38,179 --> 00:01:43,071 A lot of the people in area of democratic governance, 25 00:01:43,071 --> 00:01:49,503 participation and so on thought that technology was going to be a great asset 26 00:01:49,503 --> 00:01:52,694 for better democracies, better, 27 00:01:52,694 --> 00:01:55,705 more powerful civil society, more 28 00:01:55,705 --> 00:02:00,747 ~ fair ~ political debates. 29 00:02:00,747 --> 00:02:04,089 ~ The Arab Spring, 30 00:02:04,089 --> 00:02:07,800 the fact that the Arab Spring basically started online through 31 00:02:08,186 --> 00:02:14,586 Facebook was kind of like an eye-opener for many people. 32 00:02:14,586 --> 00:02:16,726 And I also want 33 00:02:16,726 --> 00:02:22,606 to think that that's 34 00:02:22,606 --> 00:02:28,346 to a certain extent the case. There 35 00:02:28,346 --> 00:02:34,866 are certain countries, hybrid democracies, 36 00:02:34,866 --> 00:02:35,866 democracies that are 37 00:02:36,362 --> 00:02:41,506 struggling or authoritarian countries that are like open enough 38 00:02:41,506 --> 00:02:46,929 but still authoritarian where actually technology has made that possible 39 00:02:47,070 --> 00:02:51,753 and had made possible for certain groups to challenge the government. 40 00:02:51,753 --> 00:02:57,597 ~ There is this ~ idea of the digital dilemma of dictators. 41 00:02:57,597 --> 00:03:02,640 You can't curtail completely technology because when you become North Korea, 42 00:03:02,640 --> 00:03:04,521 where... 43 00:03:06,038 --> 00:03:09,240 you lose out in comparison to anybody else. 44 00:03:09,240 --> 00:03:11,482 But if you open too much technology, 45 00:03:11,482 --> 00:03:14,704 ~ that means people have a mean to express, 46 00:03:14,704 --> 00:03:17,536 a vehicle to express, to share, 47 00:03:17,536 --> 00:03:22,570 to contact others outside. And that reduce your capacity 48 00:03:22,650 --> 00:03:28,294 to ~ control your population. So ~ that idea still stays there. 49 00:03:28,294 --> 00:03:33,438 ~ But then, well, then it was 2016. 50 00:03:33,691 --> 00:03:38,573 ~ Then it was Brexit, was the first Trump electoral victory, 51 00:03:38,573 --> 00:03:41,595 ~ the issue of disinformation, 52 00:03:41,595 --> 00:03:44,256 the manipulation of information environment, 53 00:03:44,256 --> 00:03:48,178 and I think that from there we have been on a downhill 54 00:03:48,178 --> 00:03:54,802 ~ of technology becoming not only a fundamental part 55 00:03:54,802 --> 00:03:58,764 of our democratic societies, but also one of the main challenges on how 56 00:03:58,764 --> 00:04:03,266 to deal with technology and with its impact on 57 00:04:03,586 --> 00:04:10,340 on democracy at large and specifically on how we take democratic decisions either 58 00:04:10,340 --> 00:04:13,406 through elections but also through ~ political debate. 59 00:04:20,645 --> 00:04:23,771 And I mean ideas, ideas, 60 00:04:23,771 --> 00:04:30,042 which kind of project are you working on ~ with your organization? 61 00:04:26,156 --> 00:04:29,558 So we, as an organization, 62 00:04:29,558 --> 00:04:36,211 we do mostly four things. 63 00:04:36,211 --> 00:04:39,253 We generate knowledge, we write reports, 64 00:04:39,253 --> 00:04:42,255 policy papers, policy briefs, 65 00:04:42,255 --> 00:04:47,117 sometimes manuals ~ on key issues that 66 00:04:48,258 --> 00:04:49,959 our stakeholders are demanding. 67 00:04:49,959 --> 00:04:53,821 is usually very niche things, 68 00:04:53,821 --> 00:04:55,662 very highly specialized. 69 00:04:55,936 --> 00:05:01,369 on very particular things like ~ how to organize outside 70 00:05:01,369 --> 00:05:03,300 of country voting or what 71 00:05:03,300 --> 00:05:09,024 are the federal possibilities 72 00:05:09,024 --> 00:05:12,916 for South Sudan, so the very, very highly specialized content. 73 00:05:12,916 --> 00:05:18,419 ~ We do a lot of advising 74 00:05:18,680 --> 00:05:20,921 and capacity building for governments. 75 00:05:20,921 --> 00:05:22,892 So we work hand hand with... 76 00:05:23,352 --> 00:05:26,146 governments, democratic governments all over the world, 77 00:05:26,146 --> 00:05:29,821 supporting them in different things, 78 00:05:29,821 --> 00:05:31,754 improving their parliamentary processes, 79 00:05:31,754 --> 00:05:34,398 ~ drafting new legislation. ~ 80 00:05:38,358 --> 00:05:43,299 creating new electoral processes and so on. 81 00:05:43,299 --> 00:05:48,519 We convey dialogues, we create spaces for stakeholders 82 00:05:48,519 --> 00:05:52,739 to have dialogue and to exchange ideas and learn from each other. 83 00:05:52,739 --> 00:05:56,019 And then we do advocacy. As an intergovernmental organisation, 84 00:05:56,019 --> 00:06:01,839 our advocacy is a bit different than what you would expect from organisations like 85 00:06:01,839 --> 00:06:03,439 Amnesty International or things like that. 86 00:06:03,898 --> 00:06:09,340 ~ So it's more at the multilateral level. 87 00:06:09,340 --> 00:06:15,953 We do lot of work within the UN system with other regional organizations 88 00:06:15,953 --> 00:06:20,725 like the Organization of American States or the Council of Europe 89 00:06:20,725 --> 00:06:27,648 or the African Union ~ and other spaces where 90 00:06:27,648 --> 00:06:31,640 key discussions on democratic ~ aspects are being taken. 91 00:06:39,230 --> 00:06:44,263 I was wondering like if you notice any difference in relation of 92 00:06:44,263 --> 00:06:47,415 the I mean in the different regions, 93 00:06:47,415 --> 00:06:50,658 so the different government from the different countries, 94 00:06:50,658 --> 00:06:55,485 if there is a different kind of ~ behavior or attitude, 95 00:06:52,580 --> 00:06:55,693 towards technology or towards democracy in general. 96 00:06:55,485 --> 00:06:56,485 I don't know. 97 00:07:01,939 --> 00:07:05,370 Yeah, ~ there is indeed a very, 98 00:07:01,972 --> 00:07:03,752 Yeah, I mean I can imagine for both 99 00:07:05,370 --> 00:07:10,341 very big ~ difference. ~ And it's, 100 00:07:10,341 --> 00:07:16,073 I mean, for both. When it comes to democracy, 101 00:07:16,073 --> 00:07:17,363 when it comes to democracy, it's a bit, 102 00:07:17,363 --> 00:07:22,244 this is a little bit, we don't have time for in a podcast to analyze this, 103 00:07:22,244 --> 00:07:27,426 but ~ there are different tradition and historic trajectories that 104 00:07:28,059 --> 00:07:30,280 whose regions one way or another. 105 00:07:30,280 --> 00:07:33,462 So for instance, Latin America, 106 00:07:33,462 --> 00:07:37,965 it's very aware and very, 107 00:07:37,965 --> 00:07:42,048 I would say still traumatized to a certain point of 108 00:07:42,048 --> 00:07:46,911 the story of the last century. 109 00:07:46,911 --> 00:07:50,053 And for those countries, 110 00:07:50,053 --> 00:07:54,256 as imperfect as their democracies might be sometimes, 111 00:07:54,256 --> 00:07:56,037 there is no other 112 00:07:56,545 --> 00:07:59,907 game in town. Democracy is the only possibility. 113 00:07:59,907 --> 00:08:02,188 So countries like Mexico, like Brazil, 114 00:08:02,188 --> 00:08:04,559 like Argentina, like Chile, like Peru, 115 00:08:04,559 --> 00:08:06,550 Colombia, they might have many problems, 116 00:08:06,550 --> 00:08:07,891 but there is no alternative to, 117 00:08:07,891 --> 00:08:11,113 they don't want anything else than a democracy. 118 00:08:11,113 --> 00:08:17,166 ~ The discussion is more about how to make it better and for the citizens and so on. 119 00:08:17,166 --> 00:08:23,630 think it used to be the case in Europe ~ that that is certainly that 120 00:08:23,814 --> 00:08:25,845 tide is starting to change slightly. 121 00:08:25,845 --> 00:08:30,148 It is the case in most African countries, 122 00:08:30,148 --> 00:08:35,510 but that's also starting to change and we're seeing how many Sahel countries have 123 00:08:36,191 --> 00:08:38,732 embraced ~ military dictators. 124 00:08:38,732 --> 00:08:43,655 There is like a coup d'etat type of like domino effect in the region. 125 00:08:43,655 --> 00:08:47,998 And for Asia, it's a bit more complex, 126 00:08:47,998 --> 00:08:51,000 also because Asia, it's not an homogeneous region. 127 00:08:51,000 --> 00:08:52,000 ~ 128 00:08:51,964 --> 00:08:54,756 discuss everybody together, but it's a very diverse region. 129 00:08:54,756 --> 00:08:58,648 So India, it's clearly, it's the biggest democracy in the world. 130 00:08:58,648 --> 00:09:05,171 And there's no questions about ~ the democratic system in India. 131 00:09:05,171 --> 00:09:11,795 Other regions have not had many experiences 132 00:09:11,795 --> 00:09:16,908 of proper democracy, or they keep like going in and out of democracy, 133 00:09:16,908 --> 00:09:19,889 countries like Thailand, Malaysia, 134 00:09:19,889 --> 00:09:21,350 where they have 135 00:09:21,629 --> 00:09:27,293 kind of like they have tried to create hybrid control models of democracy that 136 00:09:27,353 --> 00:09:31,575 are always in the line between 137 00:09:31,575 --> 00:09:38,440 ~ it's popular control over decision making and equality and exercising that control 138 00:09:38,460 --> 00:09:42,682 possible or not possible. It's always in that blurry line. 139 00:09:42,682 --> 00:09:47,285 And that is reflected also in the way technologies address. 140 00:09:47,285 --> 00:09:49,947 So many... 141 00:09:50,535 --> 00:09:52,346 most countries realize that... 142 00:09:52,346 --> 00:09:53,346 ~ 143 00:09:53,111 --> 00:09:58,816 A key factor to the survival or the strengthening of the democracies 144 00:09:59,216 --> 00:10:01,618 is how they manage technologies, 145 00:10:01,618 --> 00:10:03,960 how they manage the impact of technology, 146 00:10:03,960 --> 00:10:09,255 how they adapt to it. ~ But then there are different ways to relate to it. 147 00:10:09,255 --> 00:10:13,768 So European countries tend to go more into legislation. 148 00:10:13,768 --> 00:10:20,394 They want to impose rules because it's a rules-based order. 149 00:10:20,394 --> 00:10:27,368 in regarding like to the let's say digitalization that could 150 00:10:27,368 --> 00:10:34,021 be a mix of ~ how ~ because the previous question was ~ the relationship between 151 00:10:34,021 --> 00:10:38,354 the different countries in relation to could be technology or democracy. 152 00:10:38,354 --> 00:10:44,848 ~ and so I wonder to digitalization have you seen like a different ~ kind of 153 00:10:45,528 --> 00:10:48,030 Th there is ~ as I said before, 154 00:10:48,030 --> 00:10:52,113 there are some differences between ~ how regions approach this. 155 00:10:52,113 --> 00:10:58,549 ~ and ~ in general Europe tends to be more on the ~ regulatory side of things. 156 00:10:58,549 --> 00:11:03,082 They tend to try to tame 157 00:11:03,282 --> 00:11:08,577 or control what technology companies and 158 00:11:08,577 --> 00:11:13,030 the technology space ~ does through through regulation. 159 00:11:15,273 --> 00:11:17,374 It is also like a big enough market, 160 00:11:17,374 --> 00:11:20,477 so all companies will comply. 161 00:11:20,477 --> 00:11:25,450 So now we all have on our iPhones ~ USB C and that's because 162 00:11:25,450 --> 00:11:31,164 the European Union ~ forced Apple to do that. 163 00:11:31,164 --> 00:11:37,659 And ~ and they all complain but then they they they they comply with 164 00:11:37,659 --> 00:11:41,011 it because they're not going to lose the European market just for 165 00:11:41,011 --> 00:11:42,282 for the the charger. 166 00:11:43,172 --> 00:11:44,974 ~ and and because and and like this, 167 00:11:44,974 --> 00:11:47,376 this is very obvious, very visual because we can see it, 168 00:11:47,376 --> 00:11:50,208 but there are many things that ~ that go in the in the same line. 169 00:11:50,208 --> 00:11:54,482 So it it generally works. ~ it's just sometimes it's not enough. 170 00:11:54,482 --> 00:12:00,336 ~ other countries are because their legislation is weaker or their their enforcement 171 00:12:00,336 --> 00:12:02,628 is weaker, ~ their markets are smaller, 172 00:12:02,628 --> 00:12:08,333 so they know that let's say meta can threaten Colombia to live. 173 00:12:08,333 --> 00:12:10,955 because meta can survive without Colombia, 174 00:12:10,955 --> 00:12:11,955 it's it's fine. 175 00:12:12,400 --> 00:12:19,324 ~ but but any government that does something that eventually pulls meta 176 00:12:19,324 --> 00:12:24,026 out of the country will f face ~ a huge backlash from from the population. 177 00:12:24,026 --> 00:12:28,309 Because people want to keep scrolling on on Instagram. 178 00:12:28,309 --> 00:12:30,900 ~ so th they are less powerful, 179 00:12:30,900 --> 00:12:33,832 they don't have ~ infrastructure, 180 00:12:33,832 --> 00:12:36,873 governmental infrastructure like the European Union that can impose rules. 181 00:12:36,873 --> 00:12:41,756 So they they are more they they're forced to be more on 182 00:12:42,816 --> 00:12:48,558 negotiation mood and they try to find agreements and and working modalities where 183 00:12:48,558 --> 00:12:52,839 civil society, private companies and governments get together. 184 00:12:52,839 --> 00:12:55,330 ~ Some exceptions like Brazil, 185 00:12:55,330 --> 00:13:00,718 for instance, Brazil doesn't have a problem with imposing this again because it's 186 00:13:00,718 --> 00:13:03,782 a big market, so nobody wants to lose out in in Brazil, 187 00:13:03,782 --> 00:13:08,463 ~ India and so on. And and I think this is a clear ~ this is a clear signal. 188 00:13:08,463 --> 00:13:10,424 ~ Private companies 189 00:13:11,011 --> 00:13:16,083 Private tech companies today are as powerful as medium sized countries, 190 00:13:16,083 --> 00:13:20,305 and they can they can impose their will on on these countries. 191 00:13:20,305 --> 00:13:22,546 They can't impose their will on the European Union, 192 00:13:22,546 --> 00:13:25,327 on Brazil, on India, or or or the United States, 193 00:13:25,327 --> 00:13:26,327 obviously. 194 00:13:25,787 --> 00:13:32,389 ~ Indonesia, 195 00:13:32,389 --> 00:13:36,821 Australia, some some certain countries are capable of of of imposing things 196 00:13:36,821 --> 00:13:39,112 to the to tech companies, but most countries are not. 197 00:13:39,896 --> 00:13:46,882 ~ so ~ they need to ideally they should create alliances with other 198 00:13:46,882 --> 00:13:50,365 countries and impose legislation with other countries together. 199 00:13:50,365 --> 00:13:55,809 ~ so it's fine to lose out ~ on a particular African country, 200 00:13:55,809 --> 00:13:59,412 but nobody wants to lose out on the whole African ~ continent. 201 00:13:59,412 --> 00:14:04,236 ~ so that should be ~ ideally one of the ways to go, 202 00:14:04,236 --> 00:14:06,758 but that sadly doesn't happen very often. 203 00:14:06,758 --> 00:14:08,790 reaching agreements on on 204 00:14:09,036 --> 00:14:10,747 tech policy are it's extremely, 205 00:14:10,747 --> 00:14:17,469 extremely difficult. tech companies spend millions of euros in lobbying, 206 00:14:17,469 --> 00:14:23,332 which it's effective. But also there is not clarity on what this legislation should 207 00:14:23,332 --> 00:14:26,353 have. ~ so it's not that there is an agreement between countries like 208 00:14:26,353 --> 00:14:27,984 we want this particular thing, 209 00:14:27,984 --> 00:14:32,866 please do it. and there is a lot of di differing opinions on on how to go about it. 210 00:14:32,866 --> 00:14:38,448 So that also ~ makes it very difficult ~ to have this type of like regional 211 00:14:38,914 --> 00:14:39,977 pieces of legislation. 212 00:14:41,842 --> 00:14:48,316 Of course. And I w I was thinking how do you imagine like the future? 213 00:14:48,316 --> 00:14:54,959 Let's say that I mean digitalization is happening also right 214 00:14:54,959 --> 00:15:00,302 now and also in the past years and ~ I'm thinking like if in 215 00:15:00,302 --> 00:15:05,155 the future like the I mean democracy and technology are like tightened 216 00:15:05,155 --> 00:15:10,700 ~ ever more than now because it could be like a ~ 217 00:15:11,634 --> 00:15:15,296 Something that I don't know, 218 00:15:15,296 --> 00:15:21,310 an abs like a very strong centralization ~ of power or like could 219 00:15:21,310 --> 00:15:25,202 be decentralized, could be ~ so how what are 220 00:15:25,881 --> 00:15:32,806 I think there are for me there are two fundamental questions that 221 00:15:32,806 --> 00:15:39,020 will define the way technology works in society in general 222 00:15:39,020 --> 00:15:44,914 and in in our political and and systems to organize our politics. 223 00:15:44,914 --> 00:15:51,018 The first one is ~ choke points. 224 00:15:51,018 --> 00:15:54,450 who controls the tech stack? 225 00:15:55,614 --> 00:15:58,431 and who has capacity to influence the tech stack. 226 00:16:02,572 --> 00:16:06,056 There are not that many choke points, 227 00:16:06,056 --> 00:16:11,903 but there are some of them. And and and we are all very unprotected. 228 00:16:11,903 --> 00:16:16,909 ~ for those. ~ one of them is semiconductors protection. 229 00:16:16,909 --> 00:16:19,522 ~ it is enhanced in the hands of 230 00:16:21,810 --> 00:16:25,442 one country, two countries, Taiwan and South Korea mostly, 231 00:16:25,442 --> 00:16:29,284 ~ the the the production, the physical production of semiconductors. 232 00:16:29,284 --> 00:16:34,497 There is one country that controls that has well there is one company that 233 00:16:34,497 --> 00:16:40,600 can produce the ultraviolet lithography machines that 234 00:16:40,600 --> 00:16:42,851 can produce modern semiconductors. 235 00:16:42,851 --> 00:16:49,525 ~ China is trying to catch up but as they advanced the other like 236 00:16:49,525 --> 00:16:50,856 the technology also advanced. So 237 00:16:51,584 --> 00:16:58,288 Right now you can stop every I mean it it is there is a possibility that it's 238 00:16:58,288 --> 00:17:04,271 not zero that we run we we don't have the capacity 239 00:17:04,271 --> 00:17:08,583 to produce semiconductors. So that will that that's ~ if something happens 240 00:17:08,583 --> 00:17:12,806 in Taiwan or Taiwan decided to stop producing or or I don't know. 241 00:17:12,806 --> 00:17:15,217 there are many many possibilities to for that to happen. 242 00:17:15,217 --> 00:17:17,688 And that will imply we can produce cars, 243 00:17:17,688 --> 00:17:19,999 we can produce TVs, we can produce 244 00:17:21,675 --> 00:17:26,531 many things. ~ but semiconductors is not the only one. 245 00:17:26,531 --> 00:17:32,239 We also have ~ chok points in ~ in other parts of the of 246 00:17:32,239 --> 00:17:34,582 of of the global tech stack. 247 00:17:39,564 --> 00:17:44,986 The second question, and this is related to the first one, 248 00:17:44,986 --> 00:17:51,869 ~ seems to be also who is going to control the AI stack? 249 00:17:51,869 --> 00:17:58,332 And and it's related because the AI stack is part of the global tech stack. 250 00:17:58,332 --> 00:18:04,634 So who is the hardware, who's going to control for me there are like four parts 251 00:18:04,634 --> 00:18:06,895 of that tech stack. The hardware part, 252 00:18:06,895 --> 00:18:08,816 and that's the same semiconductors. 253 00:18:09,528 --> 00:18:13,869 Computing cap capacity, where ~ the data centers, 254 00:18:13,869 --> 00:18:16,190 who will own them, who will build them, 255 00:18:16,190 --> 00:18:21,171 ~ who is going to own the capacity to run the systems, 256 00:18:21,171 --> 00:18:28,073 the AI systems that by what we know now they're extremely inefficient 257 00:18:28,073 --> 00:18:31,494 in terms of energy and and computing needs and so on. 258 00:18:31,494 --> 00:18:33,881 ~ or I don't know if they're inefficient, 259 00:18:33,881 --> 00:18:37,486 I don't know if there is a technical way to do them more efficient. 260 00:18:37,486 --> 00:18:38,776 That was what DeepSeek. 261 00:18:39,160 --> 00:18:41,469 apparently was capable of doing but 262 00:18:44,086 --> 00:18:45,977 doesn't seem to be evolving in that direction. 263 00:18:45,977 --> 00:18:50,690 But there is a question on on how much how much kap computing capacity 264 00:18:50,690 --> 00:18:54,702 can we create? ~ And who is going to control that computing capacity? 265 00:18:54,702 --> 00:19:01,466 ~ Data. They are the second layer of the AI stack is about data. 266 00:19:01,466 --> 00:19:07,910 And and we have a problem with data because ~ AI is going 267 00:19:07,910 --> 00:19:10,101 to be this is a personal opinion obviously, 268 00:19:10,101 --> 00:19:11,812 but AI is going to be useful. 269 00:19:12,725 --> 00:19:18,392 Insofar the data we fit to the system to the systems are is useful data. 270 00:19:18,392 --> 00:19:22,267 So ~ and I'm not speaking about large language models that we all use 271 00:19:22,267 --> 00:19:26,682 for doing a shopping list or ~ correcting a paragraph. 272 00:19:26,682 --> 00:19:29,545 It's more about specialized AI applications. 273 00:19:29,545 --> 00:19:32,189 ~ so 274 00:19:35,842 --> 00:19:41,874 The medical data of the ~ public ~ serv public health system in Italy, 275 00:19:41,874 --> 00:19:47,357 it's incredibly valuable because that can train an AI system. 276 00:19:47,357 --> 00:19:51,909 But that's also it's also incredibly sensitive because it's not that difficult 277 00:19:51,909 --> 00:19:55,976 to I mean it's it's not anonymized probably it can't be anonymized, 278 00:19:55,976 --> 00:20:01,583 so you have to be able to link the person to the the report written by the doctor. 279 00:20:01,583 --> 00:20:04,664 so it's it's 280 00:20:05,128 --> 00:20:09,080 It's incredibly valuable because with that data you can probably 281 00:20:09,080 --> 00:20:13,371 do thousands of improvements in in medical science. 282 00:20:13,371 --> 00:20:15,192 It is also incredibly sensitive. 283 00:20:15,192 --> 00:20:19,914 And and how do we manage data and how do we control data? 284 00:20:19,914 --> 00:20:26,217 It's also part of how we are going to relate with the technology in the future. 285 00:20:26,217 --> 00:20:31,039 ~ people will be I think people will 286 00:20:31,039 --> 00:20:34,220 be very ~ against 287 00:20:34,710 --> 00:20:38,232 artificial intelligence in they f if they find out that a lot 288 00:20:38,232 --> 00:20:39,904 of their private information is there. 289 00:20:39,904 --> 00:20:45,758 ~ I I I have I sometimes have ~ feels a bit weird when 290 00:20:45,758 --> 00:20:51,171 I realize that Claude knows personal things about me. 291 00:20:51,171 --> 00:20:53,613 And it's and it's because I t I I told it, 292 00:20:53,613 --> 00:20:57,456 right? ~ But Salin is like, because you live in this part of the city. 293 00:20:57,456 --> 00:20:58,777 It's like shit, why do you know that? 294 00:20:58,777 --> 00:21:00,358 You shouldn't be knowing that. 295 00:21:00,358 --> 00:21:03,080 So 296 00:21:03,501 --> 00:21:07,224 That I think that is going to multiply in in the near future. 297 00:21:07,224 --> 00:21:12,288 ~ the third part is the foundation models. 298 00:21:12,288 --> 00:21:18,463 ~ the the and this is I don't know how the technology is going to evolve, 299 00:21:18,463 --> 00:21:23,998 but my my fear is that there is going to be a handful of foundation models. 300 00:21:23,998 --> 00:21:30,563 that are that because of economy of scale they will keep evolving faster than 301 00:21:30,563 --> 00:21:32,385 any model. It will be impossible to catch up. 302 00:21:34,005 --> 00:21:40,370 and who contr like who controls that models controls like a fundamental layer 303 00:21:40,370 --> 00:21:43,633 of the AI of the AI stack. So if you control a model, 304 00:21:43,633 --> 00:21:49,538 you're suddenly everybody's your I mean you can you have access to 305 00:21:49,538 --> 00:21:55,183 a lot of information but you can also stop supplying ~ the model to to anybody. 306 00:21:55,183 --> 00:22:00,148 So let's say you are Airbus, ~ European 307 00:22:00,534 --> 00:22:07,236 company and your AI system that helps in hundreds of ways of building planes relies 308 00:22:07,236 --> 00:22:11,827 on an American company and then that American company decides to stop supply that 309 00:22:11,987 --> 00:22:16,468 or decides to steal your designs and fit those designs to Boeing. 310 00:22:16,468 --> 00:22:19,869 ~ there are a lot of implications of of that. 311 00:22:19,869 --> 00:22:25,571 And then the last part, ~ which is the one we ~ usually are more concerned about 312 00:22:25,571 --> 00:22:29,152 and I think is the one ~ easier to solve is the application layer. 313 00:22:31,616 --> 00:22:36,839 Most of our individual experience with AI it's it's large language models. 314 00:22:36,839 --> 00:22:41,122 ~ but I think the real change of AI is going to be on 315 00:22:41,142 --> 00:22:43,263 on very specialized applications. 316 00:22:43,263 --> 00:22:47,205 So if if you are a doctor it will be the the an application that helps 317 00:22:47,205 --> 00:22:51,038 you as a doctor. that's gonna that's gonna be the the real gate changes. 318 00:22:51,038 --> 00:22:52,038 So 319 00:22:53,548 --> 00:23:00,244 This AI stack and how it's controlled by whom it's going 320 00:23:00,244 --> 00:23:03,878 to be fundamental at the geopolitical level, 321 00:23:03,878 --> 00:23:06,120 at the 322 00:23:08,414 --> 00:23:10,585 society level and at the democratic level. 323 00:23:10,585 --> 00:23:17,529 At the geopolitical level because it will create a half and half nots type 324 00:23:17,529 --> 00:23:22,852 of wall. ~ and and I I don't see 325 00:23:25,345 --> 00:23:30,469 I don't see an agreement ~ the type of agreements that we got at 326 00:23:30,469 --> 00:23:33,201 the beginning of the nineteenth and twentieth at the beginning 327 00:23:33,201 --> 00:23:36,274 of the twentieth century to like share this the 328 00:23:36,274 --> 00:23:43,059 the share the shirt a system so we can send telegrams between 329 00:23:43,059 --> 00:23:44,530 different countries and that type of thing. 330 00:23:44,530 --> 00:23:47,002 I don't see that happening. So at the geopolitical level. 331 00:23:47,002 --> 00:23:50,915 At the political level, because ~ the implications, 332 00:23:50,915 --> 00:23:53,927 the second second effects of 333 00:23:55,082 --> 00:24:00,236 of AI are gonna have a fund like fundamental reshape the way democracies 334 00:24:00,397 --> 00:24:03,819 and and political system work and at the society level as well. 335 00:24:03,819 --> 00:24:09,644 ~ I this is not my own sentence is somebody I I work with that 336 00:24:09,644 --> 00:24:13,510 say that artificial intelligence is not a communications technology, 337 00:24:13,510 --> 00:24:19,443 it's a cognitive technology. And that changes the the impact of a the impact 338 00:24:19,443 --> 00:24:23,376 of a cognitive technology is way bigger than of a communications technology. 339 00:24:23,860 --> 00:24:26,941 Even the printing press was a communications technology. 340 00:24:26,941 --> 00:24:30,342 It allows us to communicate ideas in a different way. 341 00:24:30,342 --> 00:24:36,284 And that that simple the printing press is you can trace almost 342 00:24:36,624 --> 00:24:41,825 all the evolution of society science ~ the invention of the printing press 343 00:24:42,345 --> 00:24:44,066 to the invention of the printing press. 344 00:24:44,066 --> 00:24:48,887 ~ I don't I don't believe in the hype, 345 00:24:48,887 --> 00:24:51,618 but there is there is a scenario where 346 00:24:52,140 --> 00:24:54,051 We can do the same in 300 years, 347 00:24:54,051 --> 00:24:58,523 we can do the same of linking all the evolution of the less the 348 00:24:58,523 --> 00:25:01,984 the next three hundred years to artificial intelligence and the 349 00:25:01,984 --> 00:25:05,395 way the the cognitive change that that it implies. 350 00:25:05,395 --> 00:25:12,088 ~ and I know this is this sounds a little a lot like Silicon Valley type of of hype, 351 00:25:12,088 --> 00:25:17,240 ~ just just so they can sell ~ start up and and get 352 00:25:17,240 --> 00:25:21,712 ~ stupid high valuations. ~ but there is indeed 353 00:25:22,497 --> 00:25:28,471 ~ a possibility of of the technology slowly in in a long very long term changing 354 00:25:28,471 --> 00:25:31,653 fundamentally some of these of these questions and we are not we're 355 00:25:31,653 --> 00:25:37,226 not having those those those ~ those debates we're not considering those type 356 00:25:37,226 --> 00:25:44,171 of things what happens with ~ education how do we when we have a brain that thinks 357 00:25:44,171 --> 00:25:49,514 for us how do we use our brain for i 358 00:25:49,615 --> 00:25:51,275 i I I study 359 00:25:53,066 --> 00:25:58,547 social science ~ in university in humanities and and and to 360 00:25:58,547 --> 00:26:02,339 a certain extent it gives me a little bit of pleasure to know that a 361 00:26:02,339 --> 00:26:03,699 lot of people now is saying, well, 362 00:26:03,699 --> 00:26:08,820 it's gonna be all about humanities because the the we will solve the technical part. 363 00:26:08,820 --> 00:26:13,822 ~ the medicine we don't need we don't we're not I'm ex exaggerating, 364 00:26:13,822 --> 00:26:16,742 right? But we don't need researchers on medicine. 365 00:26:16,742 --> 00:26:22,364 That can be done with AI. We need ~ people ~ with humanities 366 00:26:22,680 --> 00:26:28,135 training to know what we should be doing with this ~ cognitive potential 367 00:26:28,135 --> 00:26:30,698 we usually didn't like we used not to have. 368 00:26:32,809 --> 00:26:34,687 ~ 369 00:26:33,176 --> 00:26:34,899 It sounds very ~ 370 00:26:36,122 --> 00:26:38,506 AI optimistic but I'm I'm not. 371 00:26:40,533 --> 00:26:47,105 Yeah like I'm also quite ~ impressed when I use AI because it is impressive what 372 00:26:47,335 --> 00:26:52,535 it can do. Does and but yeah 373 00:26:52,535 --> 00:26:57,308 I'm also I also ask to myself this kind of question like ~ 374 00:26:57,308 --> 00:27:02,809 r also regarding education and ~ because now it's wonderful like to write like 375 00:27:02,809 --> 00:27:07,641 a let's say a boring email ~ maybe we can use chat GPT or whatever 376 00:27:08,573 --> 00:27:13,388 But we are used to write and ~ and so also for new generation 377 00:27:13,388 --> 00:27:15,773 is ~ something ~ 378 00:27:16,077 --> 00:27:17,077 Exactly. 379 00:27:16,477 --> 00:27:22,580 I I two two things. I I love one ~ comic stripe that 380 00:27:22,580 --> 00:27:25,341 is ~ I don't remember where it was published, 381 00:27:25,341 --> 00:27:28,972 ~ where it's ~ somebody sitting in front of a computer speaking with 382 00:27:28,972 --> 00:27:34,775 a colleague and say, See, ~ I use AI to ~ I gave 383 00:27:34,775 --> 00:27:39,207 AI one sentence and it wrote this long email. 384 00:27:39,207 --> 00:27:41,708 And then on the other side there is somebody who say, 385 00:27:41,708 --> 00:27:42,708 Look, 386 00:27:42,454 --> 00:27:47,765 I got this long email and it summarized ~ AI helped me summarize it in one sentence. 387 00:27:47,765 --> 00:27:51,336 and that process, which I mean it seems like a joke, 388 00:27:51,336 --> 00:27:56,858 but that process it's it's our it's it's fundamental. 389 00:27:56,858 --> 00:28:02,470 Like the thinking process of writing an email and actually discerning 390 00:28:02,470 --> 00:28:05,590 the meaning and understanding the what it says, 391 00:28:05,590 --> 00:28:10,902 if we don't do it, if we lose the capacity to be able to do that and and again 392 00:28:11,286 --> 00:28:17,479 writing reports and and and thinking and and and discussing ideas that's risky 393 00:28:17,680 --> 00:28:20,621 and and we need to understand what's going to happen. 394 00:28:20,621 --> 00:28:25,344 ~ when when institutions that 395 00:28:25,344 --> 00:28:30,927 are civic pillars of society like the media, 396 00:28:30,927 --> 00:28:34,549 like the universities, where new ideas come up, 397 00:28:34,549 --> 00:28:40,712 when those are hollow out by by artificial intelligence. 398 00:28:41,174 --> 00:28:42,891 What's going to happen when we don't have 399 00:28:45,740 --> 00:28:51,811 like universities. ~ what's going to happen when all of our society works through 400 00:28:51,811 --> 00:28:55,543 a handful of foundation models that can be used. 401 00:28:55,543 --> 00:29:00,304 Can be used. I don't believe I don't believe they're going to 402 00:29:00,304 --> 00:29:04,896 be salary sentinent and and them capable of like running us and this will become 403 00:29:04,896 --> 00:29:08,236 the matrix. But but there will be people that will control them. 404 00:29:08,236 --> 00:29:11,807 And what happened we will be able to to to take the Bastille? 405 00:29:14,275 --> 00:29:19,969 Do a new French revolution and and cut the head of the not not of a f 406 00:29:19,969 --> 00:29:21,370 of a person but of the machine. 407 00:29:21,370 --> 00:29:23,612 Will will we be able to do that? 408 00:29:23,612 --> 00:29:26,714 That's I think that's the question we're not putting in front of us. 409 00:29:26,714 --> 00:29:29,636 We're more worried about there's a deep fake. 410 00:29:29,636 --> 00:29:34,750 ~ but that's that that's a that's a sort of problem. 411 00:29:32,017 --> 00:29:33,017 Yeah. 412 00:29:36,573 --> 00:29:40,415 Yeah, it could be also an occasion maybe to put in discussion a lot 413 00:29:40,415 --> 00:29:42,676 of things that we are used to doing society. 414 00:29:42,676 --> 00:29:46,819 ~ maybe I'm thinking also regarding education, 415 00:29:46,819 --> 00:29:50,341 and ~ but yeah, 416 00:29:47,576 --> 00:29:48,797 Mm-hmm. Yep. 417 00:29:50,341 --> 00:29:54,713 I also saw that comic ~ the the stripe regarding the one that write the mail, 418 00:29:54,713 --> 00:29:57,245 the other one that read it and it was ~ quite 419 00:29:58,946 --> 00:30:00,201 summarize it very well, yeah. 420 00:29:59,611 --> 00:30:00,611 Yeah, yeah, 421 00:30:00,222 --> 00:30:05,856 yeah, yeah, exactly. And I mean regarding like the I mean 422 00:30:05,856 --> 00:30:11,791 who control the AI stack who control ~ also ~ as 423 00:30:11,791 --> 00:30:18,747 you say the semicon semiconductor or foundation models like what could 424 00:30:18,747 --> 00:30:25,242 be yeah I don't know a solution because I there are s also some 425 00:30:25,262 --> 00:30:28,305 ~ it's also in relation to sovereignty of data. 426 00:30:29,005 --> 00:30:34,453 ~ maybe and yeah what could be yeah 427 00:30:34,453 --> 00:30:39,119 a possible solution ~ to this because I I think or it came from the people that 428 00:30:39,119 --> 00:30:45,288 we just start to use I don't know Mastodon instead of Facebook ~ but it's hard 429 00:30:46,136 --> 00:30:47,136 Yeah, 430 00:30:46,576 --> 00:30:48,738 that that's that's that's a difficult one. 431 00:30:48,738 --> 00:30:55,083 part of it it will come from the people and I I really hope there is going 432 00:30:55,083 --> 00:30:58,346 to be a backlash against AI. 433 00:30:58,346 --> 00:31:04,011 A a positive backlash, not just people like not type of like ludits 434 00:31:04,271 --> 00:31:07,273 not wanting to use it, but like a positive backlash that people will 435 00:31:07,574 --> 00:31:13,298 the same way that apparently ~ young generations like kids 436 00:31:15,359 --> 00:31:17,621 in their like ~ sixteen, seventeen, 437 00:31:17,621 --> 00:31:23,246 eighteen years old kids are starting to have a certain backlash against mobile 438 00:31:23,246 --> 00:31:29,922 phones and they they so they organize like gatherings 439 00:31:29,922 --> 00:31:34,776 where everybody drops their phone at the entrance and then they so they kind 440 00:31:34,776 --> 00:31:36,458 of like inter interact offline. 441 00:31:37,352 --> 00:31:40,895 ~ so that's kind of they they're realizing the dangers and they don't want 442 00:31:40,895 --> 00:31:43,508 to be sitting in a room with their friends looking at the phone, 443 00:31:43,508 --> 00:31:50,424 so they they realize how to ~ work ~ against the the the machinery that forced them. 444 00:31:50,424 --> 00:31:55,049 ~ there is no one solution, 445 00:31:55,049 --> 00:31:58,882 there is no one magic bullet, silver bullet that will solve things, 446 00:31:58,882 --> 00:32:00,334 but there are things that we can do. 447 00:32:00,334 --> 00:32:02,396 ~ we 448 00:32:03,556 --> 00:32:07,197 The EU is right in moving the debate towards sovereignty. 449 00:32:07,197 --> 00:32:11,858 We need to be careful about digital sovereignty because 450 00:32:11,858 --> 00:32:17,700 ~ at the international at the multilateral level at the multilateral level, 451 00:32:17,700 --> 00:32:22,081 digital sovereignty is a term that has traditionally been used by China 452 00:32:22,141 --> 00:32:23,982 to justify 453 00:32:23,982 --> 00:32:28,843 ~ their they 454 00:32:29,063 --> 00:32:32,684 not respecting the the normal functioning of the internet. 455 00:32:33,228 --> 00:32:38,990 basically so ~ imposing firewalls basically this idea that 456 00:32:38,990 --> 00:32:43,991 the internet shouldn't be a number of connecting nodes 457 00:32:43,991 --> 00:32:49,611 but rather isolated islands controlled by each of the governments. 458 00:32:49,611 --> 00:32:56,324 ~ so but still the idea that countries 459 00:32:56,324 --> 00:33:02,706 need to have certain control upon their digital infrastructure is is correct. 460 00:33:05,571 --> 00:33:09,884 But democratic countries cannot rely on themselves like it's 461 00:33:09,884 --> 00:33:14,978 not about controlling their own stack and then hoping the rest will will 462 00:33:14,978 --> 00:33:16,359 do the same with their own stack. 463 00:33:16,359 --> 00:33:20,312 I think there is a need to have some some level of agreement among countries. 464 00:33:20,312 --> 00:33:24,075 ~ for data, for instance, 465 00:33:24,075 --> 00:33:28,559 ~ we need a bill of rights for data. 466 00:33:28,559 --> 00:33:33,643 We need more clarity. I I would even constitutionalize data rights. 467 00:33:33,643 --> 00:33:35,144 although 468 00:33:35,672 --> 00:33:39,923 constitutions are very difficult to to to get ~ an agreement. 469 00:33:39,923 --> 00:33:43,904 So constitutionalizing data rights is it's pretty difficult. 470 00:33:43,904 --> 00:33:47,635 But I I will move towards constitutionalizing data rights. 471 00:33:47,635 --> 00:33:51,775 I think we need mechanisms, I'm not a a a technical person, 472 00:33:51,775 --> 00:33:57,308 but I we need mechanisms to use sensitive data in a 473 00:33:57,308 --> 00:34:02,429 way that is respecting of privacy and and and human rights. 474 00:34:02,429 --> 00:34:04,610 And and there are ways to do that. 475 00:34:05,376 --> 00:34:10,018 Estonia has been using sensitive data for many years without problems 476 00:34:10,239 --> 00:34:16,494 and and and they there are ways to ~ host data 477 00:34:16,494 --> 00:34:22,989 in different parts and only ~ I mean it's it's an and only provide the part 478 00:34:22,989 --> 00:34:25,321 of the data that it's necessary, 479 00:34:25,321 --> 00:34:29,654 so data minimization. ~ data minimization shouldn't be just a slogan, 480 00:34:29,654 --> 00:34:30,715 it should actually work. 481 00:34:35,083 --> 00:34:36,324 And there are ways to do that, 482 00:34:36,324 --> 00:34:42,349 and I think with AI we need to be much more creative in ways to do to do that. 483 00:34:42,349 --> 00:34:47,353 I also think 484 00:34:49,313 --> 00:34:53,898 especially the European Union ~ but but also other countries 485 00:34:53,898 --> 00:34:58,481 ~ should join efforts 486 00:34:58,601 --> 00:35:02,364 to promote semi open 487 00:35:04,340 --> 00:35:08,422 AI system, semi open source AI foundational models. 488 00:35:08,422 --> 00:35:13,406 ~ you can have a fully I'm I'm very afraid 489 00:35:13,406 --> 00:35:17,188 of fully open source foundational models because 490 00:35:19,545 --> 00:35:23,468 'Cause we know that human beings are gonna find the worst way to use that. 491 00:35:23,468 --> 00:35:28,242 And be it developing a biological weapon 492 00:35:28,242 --> 00:35:33,447 or or doing non consensual porn images. 493 00:35:33,447 --> 00:35:37,640 But we will human beings will find the worst way to use that. 494 00:35:37,640 --> 00:35:43,805 We need to have safeguards. We can release completely open weight models 495 00:35:44,666 --> 00:35:46,788 into the wild, ~ because that's 496 00:35:47,506 --> 00:35:48,886 Right now that's very dangerous. 497 00:35:48,886 --> 00:35:52,007 It's it's it's it's too dangerous to do that. 498 00:35:52,007 --> 00:35:55,638 It's like if we were like if we were 499 00:35:55,638 --> 00:36:01,360 to release all chemicals that we produce in the world, 500 00:36:01,360 --> 00:36:04,721 just you could buy and ~ buy them in a shop. 501 00:36:04,721 --> 00:36:08,802 That that would be too dangerous because ~ people would do bad things with them. 502 00:36:08,802 --> 00:36:14,924 ~ but we also need to allow for certain level of open source 503 00:36:16,766 --> 00:36:21,360 in in in foundational models. So I think democracy should push 504 00:36:21,500 --> 00:36:28,045 for semi-open source foundational models in which certain aspects 505 00:36:28,286 --> 00:36:30,227 are controlled, there are safeguards, 506 00:36:30,227 --> 00:36:33,690 there are limits and there are institutions. 507 00:36:33,690 --> 00:36:40,156 ~ we have created a type of institutions like ICANN for the internet or 508 00:36:43,314 --> 00:36:47,079 UN can create ~ I don't know, there are ways to create those institutions that 509 00:36:47,079 --> 00:36:50,433 control that or ~ priv private public partnership. 510 00:36:50,433 --> 00:36:56,800 But those models that you can't create and have a logical weapon 511 00:36:56,881 --> 00:36:58,803 but will help you 512 00:37:02,232 --> 00:37:06,379 develop a system to control the traffic jumps in a city. 513 00:37:08,482 --> 00:37:14,660 That is where we that's how we can control it like that's how 514 00:37:14,660 --> 00:37:19,787 we can be sovereign with the technology because we won't depend on a single source. 515 00:37:19,787 --> 00:37:26,076 We won't depend on Anthropic or OpenAI or Google ~ model that they can always 516 00:37:29,539 --> 00:37:31,069 They they will they will do two things. 517 00:37:31,069 --> 00:37:35,300 They can always stop it. Not that they will necessarily do it, 518 00:37:35,300 --> 00:37:37,351 but the possibility is always there. 519 00:37:37,351 --> 00:37:42,530 And and and when it's a matter of developing, 520 00:37:42,530 --> 00:37:45,093 I don't know, traffic jam management system, 521 00:37:45,093 --> 00:37:51,004 ~ there is no interest. But ~ when when these systems 522 00:37:51,004 --> 00:37:55,435 are used for warfare, ~ then then then it gets ~ way more tricky. 523 00:37:55,435 --> 00:37:59,016 And then the second thing is that we know the companies tend to 524 00:38:00,414 --> 00:38:03,827 not in always improve the product but and shittify the product. 525 00:38:03,827 --> 00:38:08,861 We t they tend to eventually just try to get as much profit as possible rather than 526 00:38:08,861 --> 00:38:11,733 make the product good. And we have seen that with social media. 527 00:38:11,733 --> 00:38:13,665 They have kicked out people of social media. 528 00:38:13,665 --> 00:38:18,679 Now social media will ~ this is my my wild guess, 529 00:38:18,679 --> 00:38:21,421 but so the the time of social media has ended. 530 00:38:21,421 --> 00:38:25,335 We we don't we live in a post social media world already. 531 00:38:25,335 --> 00:38:29,468 ~ and that's because the product became ~ 532 00:38:30,356 --> 00:38:32,797 Like nobody wants to use it anymore as it was. 533 00:38:32,797 --> 00:38:35,668 So now it's a different it's a post social media world. 534 00:38:35,668 --> 00:38:40,710 So they will also do that. And and and for social media it's not it's not 535 00:38:40,710 --> 00:38:41,821 the end of the world because okay, 536 00:38:41,821 --> 00:38:44,372 fine, ~ we use other ways to communicate. 537 00:38:44,372 --> 00:38:46,543 But for a cognitive technology, 538 00:38:46,543 --> 00:38:51,465 we can allow Anthropic or Google or OpenAI to take those decisions. 539 00:38:51,465 --> 00:38:55,486 So ~ or or or Chinese providers. 540 00:38:55,486 --> 00:38:58,828 so I think that's ~ 541 00:38:59,244 --> 00:39:02,398 That's also fundamental a fundamental part of the picture. 542 00:39:04,753 --> 00:39:09,747 Yeah and I was thinking about ~ this 543 00:39:09,747 --> 00:39:14,480 ~ project ~ I don't know if you are aware of the Agentic State 544 00:39:15,962 --> 00:39:21,606 dot org and yeah what 545 00:39:21,606 --> 00:39:27,921 are if you have read it what are your thoughts if 546 00:39:27,921 --> 00:39:34,116 you think that that is a viable way ~ for a different states 547 00:39:35,551 --> 00:39:42,002 I'm I'm I'm not sufficiently familiar with ~ I I know the paper, 548 00:39:42,002 --> 00:39:48,940 ~ I know some of the authors. ~ I'm not familiar enough with with their proposal. 549 00:39:48,940 --> 00:39:54,224 ~ I know there is a lot of conversation on how the state can ~ 550 00:39:57,452 --> 00:39:59,934 hardness artificial intelligence. 551 00:39:59,934 --> 00:40:04,518 ~ and and and I will go back to what I was saying before, 552 00:40:04,518 --> 00:40:11,474 it's all about data and the trust on using data and and 553 00:40:13,538 --> 00:40:19,602 how how you manage to use data in a 554 00:40:19,602 --> 00:40:25,256 way that people trust. So ~ you know that 555 00:40:25,276 --> 00:40:29,038 by surrendering your data to a certain extent to the state, 556 00:40:29,038 --> 00:40:34,992 ~ there is no no second second degree consequences, 557 00:40:34,992 --> 00:40:36,964 negative consequences of of that. 558 00:40:39,669 --> 00:40:45,041 Yeah I I mean isn't ~ this something like 559 00:40:45,041 --> 00:40:51,153 the like as you said if an AI system in the US is used 560 00:40:51,153 --> 00:40:57,236 in Europe for the let's say something not very important then everything is okay. 561 00:40:57,236 --> 00:41:03,328 Then if there is ~ a potential whatever conflict or situation then 562 00:41:03,328 --> 00:41:07,980 of course you are dependent and you cannot really ~ 563 00:41:09,533 --> 00:41:15,396 do anything. I I wonder if also the situation for data 564 00:41:15,817 --> 00:41:21,360 is like because ~ most of the times is like this that also if 565 00:41:21,360 --> 00:41:25,701 you give data to the state nothing happens. 566 00:41:25,701 --> 00:41:31,446 but historically there are situations where the state ~ I mean the the state 567 00:41:31,446 --> 00:41:35,188 is an entity then depends who is in charge who has power 568 00:41:36,008 --> 00:41:42,803 ~ so in ~ let's say democratic society nothing will happen but if then there 569 00:41:42,803 --> 00:41:46,906 is a change ~ in the in the way the the the state 570 00:41:46,906 --> 00:41:51,709 is leaded then there could be troubles maybe, 571 00:41:51,709 --> 00:41:52,709 I don't know. 572 00:41:53,804 --> 00:41:58,648 That's and and and you say that's the fundamental part in in a democratic society. 573 00:41:58,648 --> 00:42:03,652 and that's why we have democratic societies because ~ 574 00:42:06,112 --> 00:42:09,774 I I often say some ~ democratic societies are about predictability. 575 00:42:09,774 --> 00:42:13,095 You know that that certain things can happen. 576 00:42:13,095 --> 00:42:18,538 And you know that ~ you know that you can 577 00:42:18,538 --> 00:42:23,130 get rid of a government if if need be. 578 00:42:23,130 --> 00:42:26,866 It's not that it's not that democracies elect better governments, 579 00:42:26,866 --> 00:42:30,563 it's that they can get rid of bad governments and and there is that level 580 00:42:30,563 --> 00:42:33,018 of predictability, which is basically what 581 00:42:34,038 --> 00:42:37,100 it's it's ~ predictability is what rule of law means. 582 00:42:37,100 --> 00:42:43,843 ~ there are rules so they're not going to be randomly exactly things like your 583 00:42:43,843 --> 00:42:47,936 data is and the d the government salad 584 00:42:47,936 --> 00:42:53,239 is tracing or cross crossing data 585 00:42:53,599 --> 00:42:58,842 to realise who is ~ I don't know ~ opposing the government online or that type 586 00:42:58,842 --> 00:43:02,954 of things. or or who are the I mean 587 00:43:03,394 --> 00:43:07,527 can think of worse things like ~ who are the more likely people to use 588 00:43:07,527 --> 00:43:13,641 the ~ health system the most in the next ten years and then decide that they 589 00:43:13,641 --> 00:43:15,832 are not going to get access to it because it's too expensive. 590 00:43:15,832 --> 00:43:20,345 That is doable. You know that a democracy will not allow that to happen, 591 00:43:20,345 --> 00:43:22,237 ~ because there there's a rule of law. 592 00:43:22,237 --> 00:43:26,720 There is nothing impeding a dictatorship to do that. 593 00:43:26,720 --> 00:43:32,944 ~ so you say there it's it's about having democratic societies. 594 00:43:33,245 --> 00:43:38,129 ~ it's it's not about as we say it's not about democratizing AI. 595 00:43:38,129 --> 00:43:44,044 That that is very beautiful for conferences ~ and 596 00:43:44,044 --> 00:43:47,957 and to write papers and and to speak in the international fora, 597 00:43:47,957 --> 00:43:50,159 but democratizing AI doesn't make sense. 598 00:43:50,159 --> 00:43:51,900 We don't need to democratize AI. 599 00:43:51,900 --> 00:43:55,603 We need democratic societies to manage AI. 600 00:43:55,603 --> 00:43:59,026 ~ and that's a different thing. 601 00:43:59,026 --> 00:44:01,668 ~ because 602 00:44:02,956 --> 00:44:05,177 That's what matters, that there is predictability, 603 00:44:05,177 --> 00:44:07,778 that there are rules and that we can impose those rules. 604 00:44:07,778 --> 00:44:14,102 And we won't be able to impose those rules if we don't have certain sovereignty. 605 00:44:14,102 --> 00:44:18,384 And and I think that's where ~ as I said before, 606 00:44:18,384 --> 00:44:22,806 the the concern of the European Union makes sense ~ to go that because 607 00:44:22,806 --> 00:44:24,067 if you want to impose your rules, 608 00:44:24,067 --> 00:44:26,528 you suddenly need certain sovereignty. 609 00:44:29,437 --> 00:44:36,435 Makes sense. And and yeah also regard like 610 00:44:36,435 --> 00:44:41,091 ~ before I ask you how do you imagine like 611 00:44:42,638 --> 00:44:46,761 society after digitalization. But ~ maybe the the the question could 612 00:44:46,761 --> 00:44:52,186 be reframed more like if you imagine a change ~ let's say in 613 00:44:52,186 --> 00:44:57,240 the political life in the in the way also in democratic society ~ because 614 00:44:57,240 --> 00:45:04,055 n now there is I mean ~ in most of the country when when we talk about democracies 615 00:45:04,055 --> 00:45:10,541 representative democracy and I wonder ~ if I mean because technology could also 616 00:45:10,541 --> 00:45:11,732 allow let's say a sort 617 00:45:12,370 --> 00:45:17,126 Kin maybe something like we could call it direct democracy, 618 00:45:17,126 --> 00:45:19,059 but could be also something else, 619 00:45:19,059 --> 00:45:25,728 a more like ~ hybrid sort of democracy between direct 620 00:45:25,728 --> 00:45:29,420 and representative and ~ 621 00:45:31,085 --> 00:45:32,946 I personally don't believe in those models. 622 00:45:32,946 --> 00:45:37,460 ~ for practical reasons, 623 00:45:37,460 --> 00:45:42,249 not for ideological or or or ethical reasons. 624 00:45:42,249 --> 00:45:44,196 It's for practical reasons. Ethically, 625 00:45:44,196 --> 00:45:47,959 it sounds good. ~ it's just not practical. 626 00:45:47,959 --> 00:45:52,973 And and it's not practical at the individual level, 627 00:45:52,973 --> 00:45:56,646 but it's not practical at the administrative and and governing level. 628 00:45:56,646 --> 00:45:58,818 So y we can't 629 00:45:59,894 --> 00:46:04,747 demand people to spend a huge amount of time, 630 00:46:04,747 --> 00:46:09,331 of their free time, in managing the state. 631 00:46:09,331 --> 00:46:14,589 ~ there should be spaces for consultation. 632 00:46:14,589 --> 00:46:18,738 I am all into public consultations and and and 633 00:46:18,758 --> 00:46:21,690 and reinforcing this idea 634 00:46:21,690 --> 00:46:27,554 of ~ this connecting belt between power and people. 635 00:46:27,916 --> 00:46:30,218 ~ shouldn't only be political parties. 636 00:46:30,218 --> 00:46:35,501 ~ there should be the state and the administration should listen 637 00:46:36,222 --> 00:46:38,704 to the citizens, respond to the citizens, 638 00:46:38,704 --> 00:46:44,007 media should respond to the citizens and not create manipulate 639 00:46:45,188 --> 00:46:49,761 the information environment to think to make citizens think what 640 00:46:49,761 --> 00:46:56,106 the problem is ~ so yes all into into that 641 00:46:57,730 --> 00:47:03,774 But decision making it's more complex than an individual idea. 642 00:47:06,548 --> 00:47:13,473 And there is democratic societies need to delegate into a group of people. 643 00:47:13,473 --> 00:47:19,358 You can't you can't articulate clear ideas 644 00:47:20,299 --> 00:47:22,100 on asking everybody. 645 00:47:24,202 --> 00:47:25,202 ~ 646 00:47:26,967 --> 00:47:33,360 And that's because the idea of direct democracy or like deliberative democracy 647 00:47:33,360 --> 00:47:39,383 assumes that demo that government can be a win win situation for all the society. 648 00:47:39,383 --> 00:47:40,383 And that's not true. 649 00:47:42,472 --> 00:47:46,993 Every single policy has losers and winners. 650 00:47:46,993 --> 00:47:52,345 And and politics is about who loses and 651 00:47:52,345 --> 00:47:56,156 who wins with every every ~ policy. 652 00:47:56,156 --> 00:48:00,417 Clear example making education free. 653 00:48:00,417 --> 00:48:05,199 There are losers if you make education free. 654 00:48:05,199 --> 00:48:09,890 There's people who is losing money because they invest in private education. 655 00:48:10,728 --> 00:48:16,380 ~ there is people who because of their politic ~ economic situation they have 656 00:48:16,380 --> 00:48:22,803 an advantage but ~ if you make ~ education free then they lose part 657 00:48:22,803 --> 00:48:28,405 of that advantage because they don't they had an advantage because they could access 658 00:48:28,405 --> 00:48:31,126 levels of education that now everybody can access, 659 00:48:31,126 --> 00:48:35,358 so they lose that advantage and now they have to compete with more people. 660 00:48:35,358 --> 00:48:39,600 the people trying 661 00:48:40,758 --> 00:48:47,171 religious organizations lose because they were able to control and make 662 00:48:47,171 --> 00:48:50,612 and and have an income out of education and now maybe they can. 663 00:48:50,612 --> 00:48:54,073 So there are losers with every single policy. 664 00:48:54,073 --> 00:48:57,335 And they're winners. If you make education free, 665 00:48:57,335 --> 00:49:00,436 ~ it's it's quite evident who wins. 666 00:49:00,436 --> 00:49:03,547 And politics is about who wins and who loses. 667 00:49:03,547 --> 00:49:07,679 So if we try to find always deliberate everything, 668 00:49:07,679 --> 00:49:10,280 we will always try to find 669 00:49:11,305 --> 00:49:14,887 ~ we will first of all it's very difficult to deliberate. 670 00:49:14,887 --> 00:49:21,011 There is people people with I mean deliberation is not an equal exercise. 671 00:49:21,011 --> 00:49:24,333 people have more capacity to convince, 672 00:49:24,333 --> 00:49:26,405 others are more charismatic and so on, 673 00:49:26,405 --> 00:49:28,526 and they they know ~ better to convince, 674 00:49:28,526 --> 00:49:31,048 and so there are many things that shape that. 675 00:49:31,048 --> 00:49:35,901 But you would not always find the right policy. 676 00:49:35,901 --> 00:49:40,674 ~ and there is a value also in negotiations. 677 00:49:40,824 --> 00:49:42,672 between political actors. 678 00:49:46,064 --> 00:49:52,147 I people hate this, but I think part of the work of a parliament 679 00:49:52,147 --> 00:49:56,008 is precisely that that you sit down and speak with all the political groups 680 00:49:56,008 --> 00:49:57,979 and and find agreements and say, 681 00:49:57,979 --> 00:50:02,751 like, okay, I'm gonna I'm gonna vote on favor of this law, 682 00:50:02,751 --> 00:50:05,873 and then I will get your support for this other file. 683 00:50:05,873 --> 00:50:11,195 ~ and and we exchange cars and and if if this is done ethically, 684 00:50:11,195 --> 00:50:13,196 obviously, and without 685 00:50:15,320 --> 00:50:17,981 Without going against the principles of your party, 686 00:50:17,981 --> 00:50:20,052 that's fine. That's part of the democratic game. 687 00:50:20,052 --> 00:50:22,963 And that is needed. That is that is important. 688 00:50:22,963 --> 00:50:26,695 We we we have demonized that type of behavior, 689 00:50:26,695 --> 00:50:29,816 but that is actually part of the democratic ~ arrangement. 690 00:50:29,816 --> 00:50:32,017 That's how democratic societies work. 691 00:50:32,017 --> 00:50:35,889 The agreement is at that level and not at the society level. 692 00:50:35,889 --> 00:50:41,111 At the society level, it's it's ~ it's impossible to find those those agreements. 693 00:50:41,111 --> 00:50:43,202 And then the last thing is about expertise. 694 00:50:44,784 --> 00:50:49,947 I shouldn't be asked about I shouldn't I I I shouldn't 695 00:50:49,947 --> 00:50:55,910 be consulted on waste management of the city where I live. 696 00:50:55,910 --> 00:51:00,473 Because I have no idea. I've absolutely no idea how it works. 697 00:51:00,473 --> 00:51:04,976 So there's people who have dedicated their whole life to research that and 698 00:51:04,976 --> 00:51:09,418 to to ~ study different models and how do you waste manage in the best way. 699 00:51:09,418 --> 00:51:11,940 I have no idea how it works. I 700 00:51:12,461 --> 00:51:17,343 My opinion shouldn't be that important because I don't know anything about it. 701 00:51:17,343 --> 00:51:24,286 What I might have is some political and ideological general ideas. 702 00:51:24,286 --> 00:51:29,348 Like, well, whatever you do with waste management, 703 00:51:29,348 --> 00:51:32,569 I hope it's not against the environment or it supports the environment. 704 00:51:32,569 --> 00:51:39,172 That's why there is a representative in Parliament that will bring that idea from 705 00:51:39,172 --> 00:51:40,172 me. 706 00:51:40,524 --> 00:51:44,216 But it's but I but I'm not engaging to how to organize the waste management 707 00:51:44,216 --> 00:51:46,908 of the city where I live because I I have no idea about this. 708 00:51:46,908 --> 00:51:53,151 And and my opinion is we like to say everybody's opinions count. 709 00:51:53,151 --> 00:51:55,953 No. The expert opinion counts, 710 00:51:55,953 --> 00:51:58,554 not my opinion, because I have no idea about this topic. 711 00:51:58,554 --> 00:52:02,857 So shouldn't be deciding about this this particular thing. 712 00:52:02,857 --> 00:52:06,259 And direct democracy assumes that everybody has the capacity 713 00:52:06,259 --> 00:52:08,684 to decide on everything. So 714 00:52:10,390 --> 00:52:14,402 What will happen if we were to have a direct democracy system? 715 00:52:14,402 --> 00:52:17,583 That we will become a heuristic society. 716 00:52:17,583 --> 00:52:22,525 We will just vote by by by heuristics. 717 00:52:22,525 --> 00:52:24,086 We think, okay, this is the way to vote. 718 00:52:24,086 --> 00:52:29,508 And we will eventually create what was the old system. 719 00:52:29,508 --> 00:52:31,309 We will eventually create political parties. 720 00:52:31,309 --> 00:52:38,262 We will aggregate ideas and then follow leads on what we vote. 721 00:52:39,970 --> 00:52:42,741 That's the same what we have, that's what political parties are for. 722 00:52:42,741 --> 00:52:44,882 They aggregate ideas and they say, 723 00:52:44,882 --> 00:52:50,064 I'm gonna go to parliament and I'm gonna represent your big ideas, 724 00:52:50,064 --> 00:52:53,105 your cosm cos conceptualization of the world, 725 00:52:53,105 --> 00:52:56,257 into each of the different legislations, 726 00:52:56,257 --> 00:53:01,959 be it waste management or should we ban ~ plastic bags 727 00:53:01,959 --> 00:53:04,980 or or should we build a new airport? 728 00:53:04,980 --> 00:53:08,722 I don't know, like all these small decisions where I don't have the capacity to. 729 00:53:10,008 --> 00:53:15,343 to to take ~ decision. So if we were to do it well, 730 00:53:15,343 --> 00:53:17,555 if technology because we do have the technology, 731 00:53:17,555 --> 00:53:20,278 if we were to do it well, we will end up having the same system, 732 00:53:20,278 --> 00:53:22,400 just much more complicated. 733 00:53:25,673 --> 00:53:30,765 Yeah. I'm thinking I was thinking about what 734 00:53:30,765 --> 00:53:36,998 you were saying and that as as an example you took free education 735 00:53:36,998 --> 00:53:42,460 ~ the advantage and the disadvantage for the could be for a group of people 736 00:53:42,460 --> 00:53:48,183 or for the others can also be seen in short terms and long terms because 737 00:53:49,263 --> 00:53:52,765 and this could be interesting and also 738 00:53:54,485 --> 00:53:59,529 I I'm thinking ~ because expertise is fundamental because ~ I mean also 739 00:53:59,529 --> 00:54:05,494 I don't know anything about I don't know ~ bridges but ~ 740 00:54:05,514 --> 00:54:10,328 as a citizen I could say yeah ~ the cities the the bridge there 741 00:54:10,328 --> 00:54:15,302 is not ~ maybe could be bad for environment because maybe 742 00:54:15,302 --> 00:54:17,784 my expertise is environment or something else. 743 00:54:17,784 --> 00:54:23,508 And then I was thinking f for political parties because yeah I'm very interested by 744 00:54:24,169 --> 00:54:30,755 ~ also in Italy and I always say this that in Italy ~ there 745 00:54:30,755 --> 00:54:34,978 was the five ma five star movement that was using the Russo platform. 746 00:54:34,978 --> 00:54:41,884 And ~ I was thinking that as you as you said political parties aggregate 747 00:54:44,166 --> 00:54:49,730 not ideas but yeah interests exactly. 748 00:54:47,635 --> 00:54:49,324 ~ interest, yeah. 749 00:54:49,730 --> 00:54:51,732 So could be maybe more 750 00:54:54,363 --> 00:55:00,178 a specific interest instead of because a political parti often aggregate many 751 00:55:00,679 --> 00:55:04,882 different ideas and goals. 752 00:55:04,882 --> 00:55:07,365 Yeah just 753 00:55:10,427 --> 00:55:15,731 yes. ~ this is it gets very tricky because 754 00:55:17,452 --> 00:55:19,334 you ~ the example of Italy is very good. 755 00:55:19,334 --> 00:55:25,138 You can have ~ wide broad temp parties like 756 00:55:27,500 --> 00:55:33,896 like the traditional ~ centre right and center left Democratia Cristiana 757 00:55:33,896 --> 00:55:36,488 in Italy or the PDE that they try to 758 00:55:37,954 --> 00:55:42,767 To cover a very big part of society because that's the way to win an election. 759 00:55:42,767 --> 00:55:46,889 ~ but then because they cover too many, 760 00:55:46,889 --> 00:55:50,481 ~ they can't represent them well. 761 00:55:50,481 --> 00:55:53,643 ~ but if we go to the extremes, 762 00:55:53,643 --> 00:55:56,615 then we go to like ~ single issue parties, 763 00:55:56,615 --> 00:56:01,818 where you just work on one particular thing and 764 00:56:01,898 --> 00:56:05,320 and a single issue party might be 765 00:56:06,520 --> 00:56:09,261 Good at some point if if that's a key issue. 766 00:56:09,261 --> 00:56:14,074 But then if your only ideology is one thing, 767 00:56:14,074 --> 00:56:17,116 then when you are in government or when you are in parliament, 768 00:56:17,116 --> 00:56:18,807 you have so many other things to decide. 769 00:56:18,807 --> 00:56:22,669 So w w what is your moral compass in the rest of the things? 770 00:56:22,669 --> 00:56:26,801 ~ So I I I would think 771 00:56:26,801 --> 00:56:32,595 ~ parties should use technology wisely 772 00:56:32,835 --> 00:56:34,956 to understand. 773 00:56:35,500 --> 00:56:41,902 their voters or their constituencies but also understand 774 00:56:41,902 --> 00:56:48,894 the rest of society. and and and know where people 775 00:56:48,894 --> 00:56:55,156 I mean one of the main issues why people don't don't engage in politics 776 00:56:55,156 --> 00:56:58,757 is because they don't feel represented and and and sometimes it's because 777 00:56:58,757 --> 00:57:02,038 ~ how can it be that 778 00:57:04,105 --> 00:57:08,206 one of the main if if if you ask people one of their main problems 779 00:57:08,506 --> 00:57:13,508 in for instance in Italy or in Spain where I'm from or in many countries is housing. 780 00:57:13,508 --> 00:57:17,409 And housing is not at the center of the debate in most countries. 781 00:57:17,409 --> 00:57:23,820 So people ~ understandably get angry because they feel their main 782 00:57:23,820 --> 00:57:28,662 concern in their day to day life is housing and then politicians speak about 783 00:57:28,662 --> 00:57:30,912 something else. They don't feel reflected. 784 00:57:30,912 --> 00:57:32,773 Like why are they not speaking about housing? 785 00:57:32,773 --> 00:57:33,773 This is what 786 00:57:33,591 --> 00:57:39,155 matters. so I think political parties should be using technology wisely 787 00:57:39,155 --> 00:57:43,069 to to understand these things better ~ and to talk to citizens 788 00:57:43,129 --> 00:57:46,251 on what really concerns them. And when they do, 789 00:57:46,251 --> 00:57:47,592 they are actually very successful. 790 00:57:47,592 --> 00:57:52,276 It's just ~ our information and this is ~ for another podcast 791 00:57:52,276 --> 00:57:54,728 but our information environment is broken right now. 792 00:57:54,728 --> 00:58:01,604 ~ and and they have to ~ if you try to speak about what actually matters 793 00:58:01,604 --> 00:58:02,604 to people 794 00:58:02,592 --> 00:58:07,961 you are not facing you're facing a a wall of of penetrating 795 00:58:07,961 --> 00:58:13,030 an information environment is broken and and it's not it's it's not sweet 796 00:58:13,030 --> 00:58:16,716 na right now to ~ for those messages. 797 00:58:19,646 --> 00:58:25,552 And ~ talking about messages, if you have ~ a message for the people that 798 00:58:25,552 --> 00:58:28,656 are working in the field of ~ could be democracy, 799 00:58:28,656 --> 00:58:33,642 democracy innovations or d democratic innovations 800 00:58:34,343 --> 00:58:38,167 and yeah if you have a a message for them. 801 00:58:42,868 --> 00:58:46,821 My message will be and I have always said this 802 00:58:49,612 --> 00:58:52,814 The easy sp the easy part is the technology. 803 00:58:52,814 --> 00:58:55,816 You can always find a technological solution. 804 00:58:55,816 --> 00:59:00,078 ~ the difficult part is the political will. 805 00:59:00,078 --> 00:59:05,241 is the politics of it. I I don't know, 806 00:59:05,241 --> 00:59:10,764 I I I can't. But I know it's very easy to create a super fancy tool that aggregates 807 00:59:10,764 --> 00:59:15,417 people's opinion through AI and ~ that's that's easy. 808 00:59:15,417 --> 00:59:16,778 That's doable. It's not it's not 809 00:59:17,164 --> 00:59:20,166 ro I mean it's a rocket science for me because I'm not a programmer but 810 00:59:20,266 --> 00:59:22,378 but it's for a programmer it's not rocket science. 811 00:59:22,378 --> 00:59:26,941 It's relatively easy. You can probably buy code now ~ something that more 812 00:59:26,941 --> 00:59:28,692 or less works. It's the politics. 813 00:59:28,692 --> 00:59:35,007 So I don't focus on I 814 00:59:35,007 --> 00:59:36,698 I have seen this so many times, 815 00:59:36,698 --> 00:59:42,822 just very well intentioned people focusing on the technology and showing 816 00:59:42,822 --> 00:59:45,064 how cool the technology is. But then 817 00:59:46,114 --> 00:59:50,327 they have zero political understanding and political will, 818 00:59:50,327 --> 00:59:52,988 so it never it never goes anywhere. 819 00:59:52,988 --> 00:59:56,721 There are hundreds and hundreds of tools that are super cool, 820 00:59:56,721 --> 01:00:00,513 ~ that are lying around nobody using them. 821 01:00:00,513 --> 01:00:04,396 ~ because they didn't invest enough on the political side of things. 822 01:00:06,698 --> 01:00:12,234 Thank you, Alberto. And i if you want to add something else, 823 01:00:09,272 --> 01:00:10,272 That's my pleasure. 824 01:00:12,234 --> 01:00:15,149 maybe I forgot So thank you again.