bernetta reese: good morning and welcome to the 2015 digitalgov citizen services summit. we are live here in downtown washington dc at gsa headquarters. i am bernetta reese and i am your live stream host today. first i want to say thank you to all of you tuning in and watching online today.
fabricated the new world of 3d printing pdf, be sure to follow the summit and tweet too using hashtag "digital gov '15" and if you're having any technical issues today you can also email us at digitalgovu, that's the letter u, @gsa.gov. now first i'm going to call over jacob [parsell].
jake has been our sounding board champion for the summit, and our sounding board is made up of twenty-five digital gov innovators including your's truly who all are all across the federal government and we have worked really hard to bring you an awesome event today. so jake, what can you tell us about today's event, and what can our viewers expect to see and learn about today? jacob parsell: so today's theme is open. and it's non-broad open definition today. we're going to be talking about openness in contracting, openness in hiring,
openness in content, in ways agencies can get [inaudible] anytime, anywhere, any [inaudible]. one of the things that's great about today's program is that we have twenty agencies represented in the main program. and in this expo house you can see there's about ten more agencies. it's about thirty agencies represented. and we're looking today to talk about open programming, open methods for being future ready for government like i said before. i think it's going to be a great program, it's going to be action-packed.
there's going to be a few surprises for folks out there but i'm hoping that at the end of the day that everyone in person and in the live stream will have a couple of new ideas to take back to their programs and to make government better. bernetta reese:awesome. so you see we have so much in store for you today. we also have an amazing lineup of speakers. who're we going to be hearing from jacob? jacob parsell: so, up first we're going to have white house cto megan smith.
she was kind of just wandering around here talking to us [inaudible]. one of the things we also will have is david bright talking about the internet of everything. david bright is the former communications commission's cio. always great, he's going to be talking about the internet of everything in public service. then we have [inaudible]. i'm very excited about that. we'll be talking about analytics.
one of the more interesting panels we have is about digital gov enabling. it's got a very interesting title, it has between two forums. what this is really about is when you go and talk to your enablers, your security people, your [inaudible] people, your lawyers ... we have some of those folks that got together and agreed to do a panel. [inaudible] been doing basic things in those areas to help new agencies do some very innovative things in a very open way. bernetta reese: awesome, that's really great. as you can see we are standing here in the expo.
we have a lot of agencies here who have come out to show their products and services. jacob you do public facing mobile. we have a mobile apps table. so tell us what's going on over there at the table. jacob parsell: so again, my day job i actually do a lot of mobile government. anytime anywhere i run the mobile gov program. so way over there, you can't see it, but we have our mobile gov table today and we're doing two campaigns at our table today.
first we're talking about the lost apps campaign which you can read about on digitalgov.gov today, it was a post that published this morning. we know there's a lot of mobile gov apps out there on the websites we have applications, the abc's of creative. we know there's about 250, and we think there's more than that out there. we want your help and everybody across governments to help us find these applications. you can look at how we do that online. the second thing we're doing is we're featuring a mobile crossroads testing
program. what we've been doing for the last couple of years is when abc released a mobile website, they wanted to get some usability feedback getting some compatibility testing. compatibility testing tells you if your mobile website works on multiple devices. there's tons of devices, tons of operating systems on these devices. and that's what our testing is. so all of our testers who are federal place who have done that it says,
"i've been mobile tested." i [inaudible] bernetta reese: that's right, that's right. well thank you so much jacob. well enjoy the summit today. i'll be seeing you a little bit later. now we're going to bring over one of our expo guests here today, fema, who we're really excited to talk to, that were one of the first to sign up today was jacob [nose]. so we'll let you go jacob.
jacob parsell: we'll let someone talk about the mobile app. bernetta reese: that's right, thank you jake. so we're going to bring over harry now. harry: hi. bernetta reese: hi, how are you harry, good morning. introduce yourself to the folks watching, tell who you're with and what you're here to do today. harry: i'm harry [inaudible] part of the disaster management team at fema and i'm the [inaudible] for the fema mobile app.
bernetta reese: great. so can you tell us, what is the name of your app and what does it do? harry: the name of the app is fema. it's available for ios, android, and blackberry. the purpose of the fema app is to ... bernetta reese: the fema disaster app, right? harry: it's not disaster app, it's ... it has the disaster related information. bernetta reese: okay.
harry: the purpose is to help, soon as it's prepared, for any disaster and also give them resources that are available to them after a disaster so they can recover fine. bernetta reese: how critical would you say the app has been for the mission of fema and the emergency response community in general? harry: the app has been really helpful to agencies [inaudible] phone or a smartphone or a tablet. the numbers are going everyday. fema app enables us to maximize our reach with all these [inaudible].
so people can prepare people can look at information available in the app and they can look at the resources available to them after the [inaudible]. fema is helping to [inaudible] coming from a disaster. bernetta reese: so what are some of the key features and some of the benefits if users were to go and download your app, what would they be able to do with it? any recent highlights or anything you want to mention about that? harry: yeah,
so a couple of the really cool features fema app has is the weather alerts we just launched last month. users can sign up up to five locations and based on what they've chosen in the app, they can receive alerts. the source of these alerts is the national weather service. the other feature that we have is disaster [reporter]. it's a crowd sourcing feature which enables anybody to submit disaster related information to fema, and all this content is actually [inaudible]. once this [inaudible] fema ...
we kind of moderate what they have given to us. if it's approved, we publish the content on fema.gov on the web page. the idea is to have situational awareness and also help first responders to prioritize their efforts in doing [inaudible] disaster. bernetta reese: absolutely. i can appreciate that, having worked with the first responder community at [inaudible]. so where can people go to download the app and to learn more about that.
harry: so people can go to app store for ios, play store for android, and the blackberry one for ... they can just search the key word fema and it would come up. if you want to learn more, they can go to fema.gov/app on their computers. bernetta reese: all right, well thank you so much. we'll let you go and we're going to bring over our next speaker from [inaudible]. we hope you enjoy the summit. thanks so much for talking to us.
so let's bring over meredith. hi, we have meredith stewart here. thanks for coming over to talk to us meredith. tell us a little bit about what agency you're with today, what you're here to do today, so our viewers will know. meredith s.: i am with the national archives and i'm here representing our citizen archivist dashboard which is a portal for our crowd sourcing activities. we have tagging, transcription, we have people subtitle historical videos
and they get involved online and they love looking at documents. we launched a couple of years ago but we've refreshed a lot of new tools and we're really seeing a lot of success from it. so how beneficial would you say the tool has been since you launched it, and what are folks really excited about doing? meredith s.: it's so great. people get really into it, they love looking at old records and trying to make out the handwriting and all of their contributions make an impact.
they get indexed, they go into the search, they help unlock those records, so it's really making an impact. in march we had people working for a week transcribing and we got twenty-five hundred pages done in just one week. bernetta reese: wow! meredith s.: so, people got really into it. bernetta reese: that's right, that's right and that's probably such a great impact, a great resource especially in terms of crowd sourcing,
which we want to see a lot more agencies doing these days. can you tell [inaudible] go to get more involved, to join a project ... where should they go and how do they sign up? meredith s.: sure. you can google citizen archivists. and our website is archheads.gov/citizens-archivists. all of those things, all the information is there. please get involved. again, enjoy the day and we'll stop over later if we can.
meredith s.: thank you. bernetta reese: thank you. so we're going to bring over our next guest. hi. we have jennifer here is also apart of our expo today. she is with the connect.gov table. jennifer can you tell us a little bit about connect.gov and what is the site intended for? jennifer: connect.
gov is a secure privacy enhancement [inaudible] that connects people to government. so you can access online digital government services using the connect. gov website. we leverage third party digital credentials. so that sounds like a lot of jargon, so let me tell you what it really means. bernetta reese: all right. jennifer: so you know today when you go interact with government you have a create a unique log-in id and password for every agency.
bernetta reese: yes. very cumbersome. jennifer: exactly. so my ... when i use a treasury to file my taxes, it doesn't work if i go check my benefits in ssa. but with connect.gov we provide a platform where agencies and consumers can access government approved third party credentials. so i can use my google, or i can use [inaudible] my bank.
i as a consumer get to choose how i want to interact with government. from government, we don't have to spend a lot of time [inaudible] credentials, assumed approved identity. connect.gov is an infrastructure to help leverage digital government. bernetta reese: awesome. so who's participating currently, how many agencies out there, how many folks you've had sign up if you can say that, who's involved? jennifer: yeah. so we are in the early stages,
i think the agencies that i can say at this point are va, state department, [inaudible] usda, and of course gsa, right? bernetta reese: yeah. jennifer: we're going to promote our own program, but we're talking to a bunch more. we have literally have just launched and have been available for integration two months ago. so we're live and we're rolling out, so i'm hoping to get more agencies on-board,
especially cause where the consumer used to be able to pick up, we want to have those killer killer apps. so you say you recently launched. when did you launch? jennifer: it was really just about two months ago where we have plugged in the third party credential providers so that it actually could be used by the agencies and by consumers. bernetta reese: right. so it sounds like you want to get more agencies involved and employees on-board.
how are you planning to go about doing that, and how are you planning to educate folks who don't know much about it right now? jennifer: right, absolutely. so we're just starting that effort. we got a whole marketing plan, but it starts with things like this and trying to get the word out. since we just launched, we haven't done a hard ... the hard get out the word but also it's not just education in government,
it's education in the consumer market, right? because if i'm new to government and i come and i see i can use my google, what does that mean? and is it secure, is it safe? [crosstalk]. bernetta reese: that's right. all right, well everything sounds really awesome. i'm looking forward to going and checking it out. thank you so much for talking to us, i'm going to let you go.
i'm getting word that we're getting ready to get under way with our kickoff keynote megan smith. keep watching, keep the tweets coming. again, you can tweet us at digitalgov15 or email us at digitalgovu@gsa.gov. i'll be back with you at 10:30 with sally [inaudible] from hss and we're going to turn it over to the main conference room. female: good morning! welcome! welcome to everybody here in the room. we're live? welcome to everybody here in the room and across the country,
and maybe even across the world. there's twelve-hundred people that registered to join us today. i want to thank bernetta and jake who were leading the live stream red carpet as we were opening today. my name is gwen [consten] and i'm the director of digital government and part of the best team in government. that's a team that has all of y'all innovators who are working on anywhere, anytime, and on any device, government services that information to make the government
and make the world better. thank you. today we are welcoming you with open arms, and we want to make sure that your minds are open. open: open is our theme today and we mean it very very broadly. open data? yeah. open source? sure.
open solutions? absolutely. open inter-agency efforts? open intra-agency efforts? better. today we have a great program that's built around ... that was built in the open by you and for you and our amazing sounding [inaudible]. so i want you to open the windows
and let the breeze of fresh ideas fill this room. i want you to open the door and find your way to implement these amazing ideas. i want you all to open your minds to see yourself and to see your agency in the future that we're building together. open is changing. open is now. open is new. so let's get started by welcoming [inaudible] the associate administrator here at gsa who's going to make a few remarks.
[federa] is part of the open band of teams that are all across government and she's going to open your brains. federa: well, that's an awesome intro. my role here is to introduce someone who's so admired in the government and the private sector that frankly she needs very little introduction and that's megan smith. as our government's chief technology officer, megan is leading the crazy journey that we're all on to advance the future of our nation using technology innovation and data.
it's no secret that this journey, especially with this room is not just very rewarding but can often be very difficult. megan leads this effort with boundless energy and ideation to the point where she's very affectionately referred to as our chief technology evangelist. on a personal note, this evangelism is contagious. i've even adopted a phrase that she uses when faced with the most difficult and pointed questions about how far we come and where we're going,
and that's the phrase, "we're on it." so megan, thank you for being on it. we're very lucky to have you, not just at this incredible event that this team has put together, but for leading this effort overall. and with that, welcome to the digitalgov citizen services summit. megan smith: thanks. it is so awesome to be here. i don't know if ...
how many of you guys have seen some of the stuff ... many of you built it ... have just walked the floor? i hope ... i think in the last year we're going to show people some of the things everybody's built. i just want to get a sense of who's here. so just quickly, i don't know, let's shout out some agencies. who's here?
[b. score]! dhs. sansa. yeah, [inaudible]. government publishing office! nih! [inaudible]. i heard cs tv which, you know, you love when an agency got born after the internet, so it just does the tech and we don't have to upgrade. [inaudible] state! commerce! [inaudible].
archives! yes. opm! gsa! we're in the house of gsa. it's incredibly [inaudible] with you guys. because whether you're physically here or digitally here ... we had a couple hundred people and then all of a sudden it went to twelve-hundred people are signed up, so as kathy said, are amazing gsa leader. so we a couple of hundred of us here but we really have probably about, at any given point, about a thousand of us plus we'll post this.
there's people [inaudible] i really welcome you. it's our meeting and it's our time and we need to like get a lot of things done. i was in a meeting with the co of amex with the president. he's been the chief marketing officer and the co. they have a way that they talk, they call consistency of values and consistency of reinvention. it really resonated for us because you think of the deep history of our country and the roots of where we came from into all the different walks of how we all got here and what we aspire to be?
that consistency of values is what we're trying to do. with the consistency of reinvention, and you guys represent such an incredible group of people to make this happen. we have extraordinary teammates across government. it's one of the coolest things to come ... i just got here in september. came from suffolk valley, and from sort of the tech sector of building things and it was so amazing to come into this space
and see all the things people have accomplished already and what they're trying to do. to notice that the tq i call it ... it's sort of like, think of iq and eq tech skills? tq ... was not at the main table enough. it's all over government. we the techys are all over government. we sometimes have been buried a little bit?
and then we also haven't been invited. sometimes i think a little bit about the lawyers. if you look at anybody in law, their resume, at some point they've clerked in their life, right? they might be private sector but they come in and out of government, they come and do legislative stuff. they come on the heel of their pages, they do something. the scientists, they come as [inaudible] for they flow in and out. any resume of you see of scientist,
you also see something that they've done in our government. if you look at people in sort of the communication side, i call like ... if we're the tq, the humanities q, you know, those people who can write speeches and make movies, and organize and operate, they flow in and out of government very naturally. the medical folks. cdc and nih and they're in here. sometimes in tech, and they'll be at the top of government, you would never make a policy without your top economists.
our very best economists, our very best lawyers from the american people, at the table at the principal's table. but we've sort of accidentally decided that tech was something we do pretty much through vendors or through services so the talent is kind of buried and not at the table. we're really trying to bring tq to the court table with these extraordinary teammates that we already have here. it's not that we consult stuff on our own because we never could we need them too, but it's the together thing.
so everything that i just saw out here is taking the best of you guys and merging it with the best of [inaudible] that other people are doing. one of my favorite things that we have, i don't know, we'll put it up, is the analytics dashboard. very simple tool that we did as an awesome play across government. people got the idea to [inaudible], which is a good thing the government does, policy make around requirements to put analytics on all of our websites that went out, and great heroic people started doing it. then gsa in it's amazing way, collected.
and then a bunch of people [inaudible] and other folks together, [inaudible] other cio leadership, got that over the finish line with the team here. now we have a place where you can go on analytics.usa.gov today. i think almost a third or probably about thirty, forty percent of the websites across government are [inaudible] are on our sites right now, what are they doing? i love the openness. i just learned that if you go to [@topfederalsites],
somebody noticed us doing that and you can follow somebody and see which sites are entering the top twenty. very popular sites, cross weather which is always in the top twenty, but of course where's my refund and those sites are there. the other thing that's really great and i was just thinking ... how do we think about moving ourselves to be a platform on which the american people and other people build? instead of sort of parenting the country, right? how do we flip and ...
just talking to the state department folks in the [inaudible] group, the idea of ... how many young people, i think it's about a third of many colleges, between twenty and thirty percent of most college graduates are trying to do teach for america? like huge [inaudible] whether it's the [inaudible] or the harvard team or the spelman team trying to do ... people want to serve. and so, how do we make service an opportunity you can do virtually?
so this idea of a virtual foreign service officer that these guys have launched, you can do ten hours of service and peace corps is here. maybe not everyone can go to peace corps, but what can you do as virtual peace corps? in all the things [inaudible] we write wikipedia today, how can we think in a wikipedia way as government to be the base on which we do things? we had a cabinet meeting today and we inserted the slide a little bit about the police data initiative that we
launched. i wanted to talk about the pattern of that. we are working on trying to resolve a lot of issues about policing in our country. it's the moment of our time, right now, to work on this. how do we collaborate on that? president launched this incredible twenty-eight first century task force with a great cross-section of elite americans who just know that know community organizers, people from the policing world, leadership there,
leadership from all different walks. they sat with the american people three different times with a really interesting testimony agenda to hear us speak about what the issues were and really came back with a report that had some incredible recommendations about how we could solve those problems. many of them had technology and data to them. while that was happening, some of the presidential innovation fellows noticed all of the hack-a-thons that started to emerge in our kind of world.
so how many people saw the ... for example, seattle was doing body cam hack-a-thon together with the mayor, the chief of police, and the tech community, and innovation and creative communities across seattle. did people see that happen? yeah, a lot of people saw that. the coach for america [inaudible] are incredible political protesters in the country started to activate. so lots of different talents started to shift and move
and work on the problems where the task force was consolidating any new recommendations. one of things that came out of the tech side was we noticed that there were several police chiefs who were already moving ahead or had either pilots or scaled solutions in the directions of the recommendations. so we gathered some of them together. one of the principles that i want us to think about, and you guys know this: instead of getting together and working on a problem and trying to conjure the idea of solutions ...
"hmm, we got a problem, we got poverty, we got this issue, we got to do this" .. instead, scout for somebody who has the solution already. act more like an angel investor of venture capitalists. so in this case we looked around at the police and found amazing leadership. in the case of open data, which is one of the earliest presidential innovation those who happen to be in the department of energy, but bringing kind of their twenty percent time in, they saw that dallas and los angeles chief of police had open data.
you know, years of officer involved shooting, use of force data, all this data. we're starting to open that for the purpose of transparency and innovation together collaboratively with the open communities. in the case of camden, new jersey where the president was early this week, and also in one of the carolina's two of the police chiefs were using data internally. how can they see better what was happening with their team, this is something that all kinds of people's hr teams are all around the country are using this, all around the world, to help manage their employees.
the police officers are doing it, they call it early warning systems. these guys, in addition to extraordinary community outreach and philosophy where [inaudible] we're using data in an interesting way and have really had an extraordinary impact of really dropping murder rates, dropping shooting rates, really impacting the community and really changing ... i love the camden, new jersey police lead, he said, "great policing is not just the absence of crime, it's the absence of crime with the presence of justice." his idea, the presence of justice means that people are out of their houses
and in the parks and they're apart of the whole collaborative community. so it was great to see this. the way that we did this play is we brought together those leadership chiefs together with other chiefs who were doing some interesting work too, together with the cross-collaborative community of folks like you: the data scientists, the techys, and other people, community organizers and cross-functional group. the top chiefs who already had solutions presented, and then we went into kind of a workshopping hack-a-thon around their ideas.
makes it easier to solve the problem if you already have somebody with some proven solutions that you scouted. so other people can imagine now a way to solve the thing and they will iterate and adapt and you know, take it to the 203040 version. solving things in that way. and i would imagine, not only the things we see out here, but this room is full of people that has ... you know someone who's solves some of our most pressing problems. how can we use that network of us, to source those things
and then together collaboratively bring our skills to get the solutions done whatever they are. one of the ones we've been working on also includes my brother's keeper, counsel women and girls, generation indigenous. how do we get our youth into this more fun way of working? why do we have low expectations for some kids and huge expectations for other kids? these kids are going to do first robotics and do the science fair, and these kids are going to have a summer job that's less than that?
what are we going to do to include them? so you see great people. in baltimore there's rec-to-tech, where old rec center was shut, somebody turned it into a tech center. little kids have the [nana] room, middle kids have the mega room, and in fact the high school kids made their own genius bar and now they're coaching people on how to do 3d printing and they opened a business out of it. so really amazing solutions that can come [inaudible] out of the group that you
think needs help. actually we need their help. so flip how we think and give them access and pull them into things. we were looking at with investor power at the sustainable development goals. the un will meet in september to ratify the next level. you guys know the millennium development [inaudible] so the last fifteen years of the focus of alleviating poverty, getting water, climate change, all the things we needed to do, we've thought of an idea of a flash mob of sdg,
they're called the sdgs. what about sdgx? so when the un meets and they're working on ratifying and the policy and they're going to be able to structure and move the budgets and the strategies in these ways, a group of people from across the world who already has solutions to many of the goals or pilot or the beginning, or the [inaudible] could meet together to reveal the existence of water solutions and other solutions and sort of have that happen co-collaboratively.
so i want you guys to think about executing in this way, where you go scout, get together, collaborate, and cross-functionally solve stuff and inform our policy-making teams with these great solutions at the same times. think about doing that. another one, fcc and a lot of our [inaudible] department of interior is probably here, i didn't hear that, but state department's here. why are fifteen to twenty percent of americans not online? that is crazy, right?
can you imagine not being online? that would be really hard. and today in school you can't do your homework anymore if you're not online. one of the things i noticed about [conductivity] in our crew is that our teammates, those backbone, back-call, the people not social engineers, have you ever met any in government? there's a few. we've found a few. but they're really missing from the conversation.
we didn't really realize we left them out. but they might help us if we bring them in to think architecturally. one of my favorite conversations was around ideas around school conductivity and people felt in addition to the direct work that we're doing with all of our carriers and others, when we made the internet we pulled cables ourselves and we did ... we still have an amazing high-speed research internet [inaudible] now we have internet two. why wouldn't the k-12 folks have their own amazing backbone that we could do
for a couple of hundred million dollars of some of the money that we have and get them into gigabyte internet as fast as possible in all of our classrooms instead of thirty percent would light up. we could solve the education problem if our master teachers were connected to each other, right? in lead execution. so just thinking in this way of who's missing from the conversation who can bring extraordinary heroic teamwork to solving the problem so that we can basically unlock talent.
i'll end there and then congratulating you guys on getting together. this is an incredible exciting day. it's the beginning of a lot of stuff. one of the things that i hope we work on are not only the products and services that we're serving the american people with, but also what are we doing to unlock the talent of federal talent, federal employees? how are we kind of moving forward into much better collaborative tools that we can bring and how are we really lifting our voice in those conversations
and say, "you know, i am in the meeting and this tool we're using is really retro. we got to upgrade this. let's sit down and get it done." i'll note that we always do that when we're at war, we get it done. we always go cross-functional, we get everyone in the room when we're in a war. we invented computers. did you guys see the film imitation game? yeah,
so that's [inaudible] we did the [inaudible] we did a lot of things together. so don't feel like you're at war, but think that way, think cross-functionally like, we need to get this done now and not wait, we're in a meeting. let's go get the teammates as if your life depended on it and other's lives depended and you would unlock talent on a level that we would never have seen before. imagine what we could do. so i'll leave you with that, and the charge to unlock talent.
have an awesome day. thanks guys. male: thank you megan. my name is jacob parsell, i'm the manager for programs here at gsa. i'd like to note that on twitter there is sarah [coat] from veterans affairs, she tweeted out when megan asked for the other agents. thanks for doing that sarah. up next is one of my favorite people in the government and he's laughing now. i'm not telling a joke.
his name is dr. david [bray]. he is an fcc cio, but he also comes to us today as an eisenhower fellow who got to travel to taiwan and australia. he didn't tell me if he fought any crocodiles but maybe that's part of his presentation. he will be talking about public service and the internet of everything. david, take it away.
male: thanks. hello digital gov! i have two goals in this presentation. the first you can see up there which is, "try to encapsulate the internet of everything in fifteen minutes or less. " it's going to be ultra-concentrated, but we'll do what we can. the second goal is to try to get the hashtag, "digitalgov15" trending. so if you see like here or you have some comments about the event, please tweet it cause our goal is to try to make it trending, not just in dc but for the nation as a whole.
so in fifteen minutes or less: first i want to talk about what i mean by public service, because if you notice i am using the word public service because i think it includes both public professionals, but also first and foremost the public, which we are all members of. as well as the private sector. so i want to actually sort of encourage us to think about, yes we're digitalgov, but at the end of the day we're really about digital public service.
that includes multiple partners when we think about what we the people choose to do together. second thing is, i want to actually step back a little bit. before we talk about the future and the internet of everything, let's go back to 1969. so this is a map of the united states and it really is basically when they were all of six [notes] on the internet at the time. we were in the southwest portion of the united states.
we've fast-forward, you can get to 1982, by the time you get to 1982 you got the united states has some connectivity. we got a transatlantic and transpacific cable, but it's when we get to 1993 that things get really interesting. you see, there's actually a good news story here and not so good news story. a good news story is, the united states and new york pretty much well-connected. rest of the world we're getting connected but not there yet. but there's also a flip side of that too which is, congratulations that means just about everywhere in the world is also connected
to the united states as well. there's interesting ramifications of that. come 2007, the map actually gets so crowded that they can't actually do borders anymore. you can still make out the outlines of countries based on the map but the borders are actually gone. by the time you get to 2010, to visualize the internet they gave up actually trying to show it on a geographical map.
this is actually the state of the internet and what it looked like in 2010. so i'm talking about the internet of everything. i want to highlight that because i also know people also reference the internet of things and this is at least my own personal view, which is it's really about when everything, almost everything is connected. cause right now i would say the internet of things is already here. [inaudible] also came up and it was said to me multiple times in taiwan and australia as well is, which is, "who's going to address these hard long-term questions?
" this was said to me actually by several political figures in both countries because they recognized and their terms are short and that these issues will not just be solved in just a year. they probably won't even be solved in three or four years. this is going to require long-term thinking. so for those of you in public service, whether you chosen to do it as an interesting brief stint before you go on to something else, or you're more committed to the long haul, i would say ultimately it's going to have to be us.
industry is going to focus on for-profit, and that's great, that's what they should do. politicians are going to focus on the short-term because that's what they should do as well for elections. it is going to fall on our shoulders to think about, what is the long-term? what is the world we want to live in in 2025? how do we start preparing for it now? the other thing that i also want to point out is, your world of change agents is needed now more than ever.
by change agents i mean individuals willing to brave and change the status quo. i'm asking you if you're willing to be a change agent, and hopefully because you're here today you are, two things. i have seen cases in which people have tried to bring change without first understanding the existing narratives of the organizations they want to change. i would highly encourage you to seek the philosophy of "seek to listen and understand first before you seek to be listened to and understood. " that is essential to being a change agent. second, have a strategy for whatever friction your change will produce.
that's the reason why i have this [inaudible] because leadership comes from the greek word [inaudible], which means to send unto death. if you're going to be leader, you're going to be stepping out and facing expectations being disrupted. you're going to face friction. and you need to have a strategy to not die. nelson mandela's most interesting comment was said his most trying moment was when he actually turned to his own party, the anc, and said, "we're going to make peace with the white minority.
" he almost had a [inaudible]. it's worth considering is that [inaudible] when he said we're going to make peace with the palestinians, it was actually a fellow israeli that assassinated them. if you're going to be a leader be mindful of how much change the different people you're trying to lead can tolerate [inaudible] strategy for that friction because i don't want any of you to die. we need to actually have this [inaudible]. so, that said for my remaining half of my talk,
i decided i would at least try and take a risk too. how many people know the song alphabet aerobics? we have a few people. for those who don't, it's actually a rap that actually goes from a to z. now, i'm taking a little bit of a risk, one because i can't rap and i don't think i would try to rap. then two i'm going to try to make some future predictions from a to z about the internet of everything. we all know trying to make future predictions is probably a bad game if you try
to go anywhere past two or three years. the future probably will be different. with that said, the remaining half i'm going to give you the internet of everything a-z. so let's began. always on access. accessibility, affordability, we must address the digital divide. bring your own blank: be it ear pieces, glasses, or watches. what we wear will be online.
see as careful caution, correlations cross connected data. crucial we protect privacy. d is data deluge. global data in 2020 should exceed all words humanity ever said. exponential everywhere, everyone, everything. from fourteen billion devices now, in five years fifty billion or more around the world. g is great opportunities, greater results, and government reinvented. h is how best to engage the public.
how best with local government. how best to help each other. i it is important to include all perspectives and initiatives including those who opt out of internet use. just because our data sources can be joined, doesn't mean we should. joint discussions with the public on privacy by design. key to keeping in mind the public will regularly be mined. we must serve benevolently and kind, letting anyone reuse our code, license liberties, or promote more trust.
leverage future reused across efforts. makers movement means more can manufacture mass fabricated tools affordably. notions of what nations solely did, near future made change. new tech is about networks. open source and security, open design and apis, open choices for us all. promoting positive change perhaps the perplexing part. legacy processes future depends on purposeful transformations. quite simply tech changes how we organize, quite the question, how will we? r is raising questions regarding the future of security.
rational online discussions, representative democracy. s is scale of change since it's not linear it's exponential search by keywords one day will no longer work. t is trying to tell what is important in the data deluge: technics, tools and tactics transform how we work. ultimately it's about uses, use by humans trusting humans and use by machines trusting machines. v is volume, velocity, variety, volubility, veracity. with the coming future we're going where we've never been,
we would need new ways to adapt. x is exponential experiments to exchange expertise. y is you matter more than ever. you can help transform our public service, you can be positive change agents. zeros and ones, it's much more than that. zooming towards the future while remaining true to we the people. thank you very much. be change agents. male: so i was checking,
i don't think anybody tweeted out the alphabet but i figured somebody's working on that so get on that. a bunch of good tweeting there. thanks to david for that great presentation. [inaudible] it was like this was going to be very ultra-concentrated and he did it, so thank you david. there's this program called the presidential innovation fellows program, they are apart of it, some of you. well we're blessed today to have somebody all the way from boston.
mya hanson, who i asked if she had a boston accent, she said, "sorry i don't. " but she is from boston, i can confirm that. she works for noah and she will be talking about unleashing the power of big data through public private collaborations. mya, take it away. female: thank you. thank you all. is my mike working?
can the people in the [inaudible]. so first of all, boston really isn't that far away, it's actually pretty close. i go up there all the time, so the folks who are here from san francisco that's a long ways away. i'm going to start with one of those annoying "how many people in the audience" questions. how many people in the audience in the last six months let's say, have downloaded noah data? okay good, we got a couple.
how many of you in the last six days have checked the weather forecast: from weather.com, accuweather, weather underground, seen it on the local news ... so that is everyone, right. so that means you've downloaded noah data. all of the big weather companies use noah data. noah's mission literally stretches from the bottom of the ocean to the surface of the sun. they do weather forecasting, they protect lives and property by doing warnings and alerts for severe storms, ocean exploration,
they do nautical charts. they do space weather predictions for the aviation and radio industries. so if there's a big magnetic storm or solar flare coming that's actually kind of a big deal for satellites. so they do a lot of stuff and they have a lot of data. so i'm a presidential innovation fellow and i'm here for a year helping them out for something called their big data project. the basic gist of it, as mentioned, noah has a lot of data.
they do severe storm warnings, coastal mapping, weather and climate models, satellite imagery, fisheries data, i forgot fisheries. so they run the national marine fisheries and they do like stock protection and marine mammal protection. they help fisher people find stocks, they help fisher people figure out where they're going to avoid by-catch when they are catching things they're not supposed to be catching, stuff like that. surface observations a lot of data, right? no really, a lot of data.
so this graph which is impossible to see, i can't even read it, basically says that by 2030 there are going to be nearly 300 petabytes of total data that noah has stored in it's archives. that's just the archives. they don't archive everything, some of like the model generation, the weather model generation just kind of ... they use, they make their forecast, and it moves on. that's actually useful data too, but there's just, if not possible to archive everything,
it's not possible to get everything out and make it available as part of the open data project that the [inaudible] 1313 from the president and getting data out to the public. which noah really believes in. they have been at the forefront of open data forever. one thing about this graph, if you tried to stream this data over the next fifteen years across the internet you're looking at approximately five megabits per second. so that's not that much, your home connection is probably fifty at least.
but then, if you think about all of the different industries that need to use this data, and you multiply that five negatives per second by the number of people who are trying to download it, that's an incredible amount of bandwidth. there are thousands of people who are trying to download noah data, even more when it's a severe weather day. so this is a really big deal. it effects noah's mission. they're hitting noah's systems.
so if it's bad weather and people are trying to download data and noah can't give their data across the network? that's a huge problem. that's not okay. so it needs lots of infrastructure and noah has lots of infrastructure. this is their end-way of network and they are all over the place and this is crazy fast network connection that's internal to noah. they're able to ... they're going to cover their own usage but allowing everyone else to get at
this data is hard. it also costs lots of money, and that's your money, so noah is obviously an agency, federally funded. so there's a lot of money required to build up this digital infrastructure to distribute all of noah's data. so what do we do? noah doesn't have enough money, you don't have enough money to pay for the all infrastructure that's needed. the idea that the folks at noah came up with is that if we have all of these
industries who are using noah data and are making money off of noah data, and noah is delivering the data to them, is there some way that we can [inaudible] that hey this data they you're using that you're getting is so valuable to you. you're making so much money off of it. could you maybe kick in a little bit of that money to help us kind of fund the infrastructure to get it out to you? and oh by the way, we have a lot more data that we haven't been able to get out to you
and if you could help us out here a little bit, we could make that available to you. you'll make more money. right? everyone wins. so noah released a request for information the beginning of last year asking the industry this question. the response was really positive, everyone said yeah that sounds awesome we love your data, we want more of it.
but we're not sure how this whole kickback, we give you some money thing works, it's sounds a little sketchy [inaudible] and we're not really sure how it works either. so is there ... how can we figure this out? and who are the right people to do this? you know if industry is going to help us out with this, maybe they could help us redistribute the data so if we get it to them they could redistribute it to everyone else
and kind of force multiply noah's infrastructure. they were interested in that idea, they thought that was kind of cool, but again you know, the weather companies for instance, who were most interested in the data, they weren't all that interested in also making it available to their competitors. so that didn't go very far. and so we thought and we talked and talked and we said, okay, we need infrastructure.
who has infrastructure? oh the infrastructure is the service providers. so we thought, all right amazon, google, ibm, microsoft, the open cloud [consortium] of the university of chicago whose big in the research industry: these folks have lots of infrastructure. maybe they might be able to help us with this problem of we need to be able to get this data out there. we included them in the request for information, we included everyone in the request for information, we held an industry day,
we talked with folks. huge interest, over 300 people came to this industry day from 200 different companies. again, there were still just lots of questions about how is this going to work exactly? what do we do? so, what we decided to do was approach this as a research and development agreement. about a month ago, in fact exactly a month ago,
we announced that we have come to an agreement with each of these companies, amazon, google, ibm, microsoft, and the open cloud [consortium] under a collaborative research and development agreement, corroborative research and development agreement or [inaudible]. so that agreement allows us to work with them to try to figure out how we do this. both how do we do this technically, to how do we get the bits from noah out into the cloud.
also, how did they do this from a business model perspective. so the data is public data, right? it's public domain data. they can't charge for the data. it's not legal. they can charge for things like shipping and handling of the data but, you know, that's weird so how would they make their money? the premise is that these guys up here, the folks who use the data are going to come to these cloud companies
and want to do compute against the data. that'll do the compute on the company's infrastructure and that process will bring more customers to them and will allow them to make more money which will help justify their decision to help us make this data publicly available. this is the experiment that we're trying with them, we're about a month into it. you can basically think of it this way, data is sand. noah has lots of sand, right? the cloud providers, they have a sand box.
so hey, we can put the sand in there. they also have tools to play with the sand and so if we give them the sand, they put it in their sandbox, everyone can come and play in the sandbox. they also have these tools that people can use to play in the sandbox and so the private sector can come and play and build all these really incredibly cool things. also the world can get at the data, and everyone can get at this data and benefit and people can do things like find out that there's a big fan storm coming in
the saudi arabia peninsula. so it's a win-win situation for everyone, or at least that's what we hope right? but where we are now is this is a research and development project. we are working with the cloud providers to try to figure this out so we're hoping to start with a prototype. we are looking at some data sets that maybe haven't been available up till now or haven't been easy to get to. we have some radar data sets for instance, archived radar data sets where you literally cannot download twenty-four hours
worth of data within twenty-four hours, so you're never able really to catch up. just because it's so big and there is unlimited amount of pipe, right. so we're working with the cloud providers to figure out what are the interesting data sets for their customers as well as for the noah customers and how do we get it to them in a fast efficient way because some of this data is time-sensitive. they're working with their partners with their customers to form what they're calling data alliances. so if you have a cloud provider here,
what we're working with them to do is form effectively a miniature eco-system around that cloud provider so that they can prove out this business model of if we distribute data they would come and use it. each of the cloud providers has around them something called the data alliance that's made of their customers and other interested parties, people who are joining multiple of these alliances. they're talking amongst each other to understand how the other alliances are working. so we're really excited about this, right?
this is a cool research project, this is a cool experiment where each of these data alliances is effectively acting as a prototype for the entire market. because there are multiple of them there's market forces acting on them. we can really figure out is this business model going to work? i'm not sure what's going on over here but it's kind of jazzy. i'm liking it? is that my five-minute warning? [inaudible].
here, i think that did it. it was my fault, i pressed the volume button. anyway, i think that was pretty much it. that was my walk-off music. but i'll be around and i'm happy to talk more to the folks. i know we're not doing questions here but i'll be able to talk more with folks afterwards if have questions about how this works in practice or you want to be connected with any of these data alliances. if you're actually interested in being apart of this project
and getting involved with it, i would be happy to connect you with folks. thank you all very much for listening and having me. male: thank you mya. that was a great presentation and we got a little jazz at the end so that's always a good fun time. yeah, jazz [inaudible] here in the front row just so you know. if there's some folks standing up in back, there's actually some seats here if you'd like to come up. i do have a very fun announcement,
we are trending on twitter right now with the digital15gov hashtag so thank you very much. thank you for those of you who are on the live stream telling us about anything you're noticing or any issues you're having, so we appreciate that and we're addressing them as you send them to us. now we actually have our first panel of the day, we have one of two official panels and i'd like to introduce the federal wide analytics program manager marina fox. she is going to be leading this panel.
come up marina and the rest of your panel. leveraging open and analytics including surveys, web analytics, and campaigns to improve customer experience. marina panel take it away. marina fox: morning everyone. ready to talk some data? okay. somebody is ready, that's good. all right, so today's talk is going to be all about leveraging the data,
surveys, campaigns, web analytics, to drive decisions. everybody has data, but this talk is actually going to be about how to leverage the data and make those decisions. today we have a great panel here. we've brought you the experts to answer some of the tough questions. so let me just introduce them: sam bronson sitting next to me is the digital analysis manager for the health and human services. he currently leads efforts to develop visual data of various data sources to
inform decision making. sam previously served as the manager of the environmental protection agency's web analytics program where he helped modernize that program's software and the analytical capabilities. sam has given training webinars for digitalgov and have lots of expertise in many digital analytics tools, include google analytics. welcome sam. dr.
david cooper is a psychologist with a mobile health program at the national center for telehealth and technology- part of dod. he serves a program manager for mobile apps, related to issues around military and veteran psychological health. cooper leads a team of software developers, usability researchers, graphic designers, to create, develop and deploy mobile apps and web-based apps. david also acts as a subject matter expert in the use of technology in conjunction with the treatment of psychological health issues. before that david worked as a clinician in the field of neuro-psychology
and neuro-rehabilitation. welcome. last but not least, anne henderson. anne is the new media web content manager at the pension benefit guaranteed corporation. anne oversees and manages the content, appearance, and functionality of pbgc's internal and external communications tools. she serves as the face of the organization's website to employees and customers and engages at all levels to improve
and promote communications inside and outside the agency. anne is responsible for website governance, communication resources, standards, analytics, plain language and web policy. welcome anne. all right, so let's just jump right in. a couple of months ago i was reading a study published by forbes that this year in 2015 organizations will be spending a lot of money, in fact there are a bunch of us dominated by analytics related initiatives. i think on average we're 7.4 million dollars,
this year alone that the enterprise will be spending on data related spending. that alone didn't really surprise me because, well let's face it, analytics is a hot topic, everybody's talking about it. we're all in a way, analysis these days. i mean, think about it: who doesn't know how to use excel or run a report in a web-based application. but what kind of caught my attention was the fact that that spending was specific for data interpretation, taking that data to the next level. the reason we know that is because forbes actually asked those companies,
"have you already deployed your big data analytics projects or policing the process of it?" and they said "yes. " so they already invested a significant amount of money on building the infrastructure behind it on collecting the data. so this was not about just collecting the data, this was also about using the data to drive decisions to improve customer experience. so it really resonated because this is where we are as the government as well. agencies collect tons of data.
i mean, listening to noah alone we all have a lot of data. the government wide analytics program, the digital analytics program alone is collecting data across thousands of websites and alone we collect 1. 6 billion hits per month across all those websites. so we have a lot of data. agencies have a lot of data, we have survey data, we have usability research, we have campaign data, but what do we with it together? how do we use, how do we integrate at all to drive decisions?
because one data source is just a data point alone taken out of context. you can't really use that. how do you bring it all together, how do you tell a story, how do you drive the decision and make a change? so that's what we're going to be talking about today. with that, my first question is for anne. taking kind of a broad approach, in your opinion in your expertise, what analytics are essential when it comes to delivering digital services for the government and why?
anne: thank you marina. i'll say something that i know we've all heard is that you can't manage what you don't measure, right? so i think we all need to have set up at least the kind of basic suite of tools that we could use to measure the success of our outreach to our customers and to citizens. also, be specifically to us is the pension benefit guarantee corporation, pbgc, we use for our web tool. we have the adobe [inaudible] whatever they're calling it these days,
and we set that up with specific kpis. we set key performance indicators so that we identify certain customers based on their behavior on our site and set metrics for them. for example, our pension plan practitioners, we identify them by the certain sections they visit on our site and then we say, "we want them to sign up for our email for their ... what's news for them or we want our workers or retirees to sign up for our blog updates. " so that's when we would use that tool.
we also use the gov delivery email tool, we've been really active in using their campaigns and their reporting. we follow very closely things like open rates and click rates on gov delivery. we're also big users of the [4c] answers tool, which is the customer satisfaction survey, i think a lot of you might use that too. we actually have a bunch of different surveys, i think we have four total our website. some are applications, some on the main site.
we also participate in [inaudible] and send our information to them. we also participate in the usa search program. one of the things i love is when folks look at their search data, see what people are actually coming in and searching for. not necessarily what we think people are searching for, and use that to help frame especially the content that goes on the front page of their site. so that's something that we're working on. i think where it gets interesting is when you take the suite of tools
and you start integrating them. so for example, we've integrated our adobe and our 4c, they talk to each other so that we can look at our adobe data when 4c's customers scores come into that and we can say, "wow, when people are on this kind of webpage they're really satisfied. but when they're on this kind of webpage, they're really not. so what can we do to take those pages on our site that has some of our lowest customer satisfaction scores and increase them?" we also do that [inaudible] campaigns with the gov delivery campaigns.
we send out the information, we tag it with a campaign id and then that we can track that not only in the adobe tools so i can see for last month what were our most popular campaigns, but i can also track that in the 4c so i can see the people i sent this specific email to, i can see their relative levels of satisfaction based on the folks who just came to the websites organically. so i think a lot of it has to do with making sure you have everything set up properly and being able to look at the big picture of all of it.
i know a lot of our leadership doesn't have the time to get into the nitty gritty of the data like we like to do and being able to have all of these tools talk to each other. at least for us at pbgc has been really helpful with getting those numbers and that information up to leadership so that they can support some of our initiatives. marina fox: great. thank you so much, i think this was a really good point where obviously anne mentioned a lot of
tools. she talked about campaigns, she talked about gov delivery, she talked about surveys, answers ... a lot of information right? anne: there's a lot. marina fox: again, we have a lot of data. how do we bring it together? sam can you speak to your experiences as far as the process. like how do you go about picking and choosing what makes sense
and is there a roadmap or checklist? sam: sure. anne mentioned kpis, i think the process is of the utmost importance so defining your goals first and foremost when people come to you and say, "i want to track these metrics. " it's always pushing back and saying, "no way, [inaudible]" and that process can be [inaudible] and take some time. but getting those goals, setting your kpis and having a hypothesis. really you're testing something and you expect a change ...
you have to have action around that. could be a web redesign, could be moving things around a page and testing the placement of certain things, it could be the content of your blogs or digital campaigning but you're always testing something always have something in mind. that's opposed to just looking at it, seeing something that happens and then asking why. but we don't really know why because we didn't have hypothesis.
that suite of tools is just that, it's a suite of tools when formed through a hypothesis. so it could be voice to customer, data one day, could be web trafficking another, could be call center data, another. it's really just having that goal kpi, hypothesis and action process nailed down and reaching into those tools including that government wide programming. you could be using that for ... government is sort of a unique space so there's certain things that don't
happen in private sector the same way they happen here. could be you want to level check your developer portal against other agency's developer portal or your blog against theirs keeping context in mind. you're not going to compare regulatory blog to like a nasa blog, but you know, just knowing what the questions are, knowing what the process is, and then sort of reaching into that suite of tools as necessary. so building on that, talking about the shared analytic sources such as government wide digital analytics program, i'm curious david,
especially using your psychologist background, how can we leverage that data across the government to really get into the minds of the consumer, to really understand how to make them happier and how to serve them in the best way that truly helps them come back and they want to come back. david: right, it's important to remember when we're talking about analytics what is the purpose. you know, when we're talking about products, a product is like a garden.
you can't just plant it and then let it go. you have to constantly use care and maintenance to maintain and feed that garden, so you constantly have to be optimizing it for your users. analytics shouldn't be a static reporting tool that you're just letting your upper levels know how this thing you did is going. it should be used to help improve it as you go along to making your sites stickier. understanding, doing funnel diagrams, understanding where you're losing users in the process of your site
and how then can you optimize your site to make it stickier. taking lessons from behavioral economics about things like choice theory. if you have too many choices, how can you simplify those choices, how can you highlight the choice that you most want people to choose and then use experiments to make sure that your design changes are having the intended effect. really it's about ... again, i'm glad you said talk about the kpis and the behavioral diagrams because that's really what analytics affords us in
terms of psychology is a way to gather data from the thousands of users that are visiting our site because we can't go up to every single one, we use this data to help us understand how they're using our site. then finally analytics shouldn't necessarily be a sub-replacement to talking with actual users. so things like 4c and things like that that allows you to actually drill down and ask us specific question, "okay why are we losing you here. what can we do better and how can we better understand your experience?
" are all important uses and practical applications to [inaudible] lessons from cognitive psychology, behavioral economics to use analytics for our users. marina fox: i think those are great points and i think what i'm hearing is that having all this data is great, but really when you approach it you have to have a goal in mind, which is why oftentimes especially in the [inaudible] we have people who are so excited to have access to all this data. once they have access they come back to us and say, "what do i do with it all?
it's great you know, it's overwhelmingly great, but how do i began to ... " you know, our human brain can only process so much information at a time, so we really have to structure it to structure our approach to it to ensure that we can actually handle it. putting it in the [kpi] perspective going back to what is the purpose of this website? why do we have it in the first place? does it make sense the way we designed it because maybe we'll have it for one thing but people come for something else.
you know, let's capitalize on that or let's take a really good look and see if we have to go back to the drawing board. so i think that having the kpis are really a helpful way to structure the thinking when it comes to driving success. you don't know how to measure success if you don't know what the kpis are, and that's the bottom line. all right, so i think this is a great discussion but so far we've kind of talked about things in a really broad sense, so let's take it down a notch
and talk about some concrete examples. mike my first question is, back to anne, speaking pbgc can you think of a good example or situation where you were able to take different types of analytics and successfully drive a change or success to bring a change to your site. anne: absolutely, we had a really good example of this relatively recently. pbgc, i won't hold it against you if you never heard of our agency before. we insure pensions for about forty-million people, we pay pensions to about 1. 5 million people, and ideally if you're in a defined benefit pension plan
and we insure you, we kind of want you to go through your whole life and never even know we exist. but for those people who we actually pay their benefits every month, it's really important that they keep in touch with us and that they know what we're doing and what we're up to. we use to have a printed newsletter up until relatively recently that we would mail out to those 1.5 million customers so about two years ago, we started using gov delivery to send out a link to the pdf with that newsletter with kind of information for what folks are going to find in it.
as i was looking at some of the data that was coming back from that, i was astounded at the results that were coming in not only from adobe and from the open rates and some of the numbers we were seeing in gov delivery, but the fc customer satisfaction results were amazing, and the stuff that people were saying were all pointing to, "hey you guys need to communicate with us more, we want to hear from you more. it might not be specifically about me or my personal benefits, but we want to keep these lines of communication open." so i was able to take some of that beginning data and go to our leadership
and say, "hey, we need some resources to do this better. we need a proper email, quarterly newsletter, we need to take information from our blog which was relatively recently created, and we need to be able to send that out in a better kind of larger way. i was able to get permission. i essentially walked around the building and i said, "every email you have, and every database you have, i want everything." we pulled everything together and we created this thing called retirement recap and we sent that out.
it was a collection of blog entries among other things. we did all the campaigning, we set up the campaign idea and gov delivery, we got everything set up in adobe so that we were all ready to go. we kind of warned everybody that this was happening. we're a small agency with a small amount of web traffic. we sent this out to about half a million people. we did [inaudible] to help our call center a little provided calls come in. the response we got was amazing, it more than quadrupled our web traffic for that day.
the response activity in 4c, the number of responses that we got in the 72-hours following sending that email was as many as the number of responses we got in the past seventy days. the score for those past seventy days were something like seventy-three. the score for those 72-hours after sending it was eighty-five. so we were able to really see some amazing results from that. and it was something that took some resources. you know, i needed to be able to put some other things aside to be able to focus on that,
and that's something that we were able to get support from leadership once we were able to show some of those numbers. they really liked seeing things like, "hey, this is have a forty-four percent open rate for these half million people, that's really really high. this shows that our customers want to engage with us, they trust the kind of information that they're giving to us." one of the kpis on that was to get new people to sign up for the blog. we got over a thousand people to sign up for the blog in like the two days
after we sent it. so those were some of the really great numbers that we had in response to that. so creativity pays off. walking around the building ... anne: i created more work for myself. marina fox: reaching out, yes. that's awesome. as i was listening i remember one example where how important it is to be creative and not to trust one number you see.
just because you see a spike in page doesn't mean people like it. you have to challenge yourself from thinking, "let's check another source. let's make that we're not you know, delusional here." my favorite example is, back in the day when i worked for aol, we had a spike in one of our site's page views. [inaudible] wow, people for some reason all of a sudden are interested in looking at our data. what happened? people came up with all sorts of theories, none of them were true.
the reality was, the page performance was really poor, so people kept hitting their reload button and the page would load halfway and then there was an ad and it would just basically prevent the rest of the page from loading. but it was registering as a hit, it was registering as a page view, because it was partially loaded. the moral of the story is don't trust one source of data. always make sure that you combine it, you leverage your analytics what you have to ensure that what you see is true
because again, a number taken out of context is just that, it's a number. thank you anne for sharing your story and i'm glad that ... i keep sharing my story because i think it just wakes people up to be like, "yeah, you're right. you got to make sure you check your sources. make sure you're not trusting just trusting one source of data but you leverage your analytics to tell a full story." anne: our quick story about that is our three-hundred times spike in data. we all came in one morning and said, "oh my goodness,
there were three-hundred times more people on our site yesterday. " that's what happens when bill o'reilly talks about the government owing you money and going to pbgc.gov. go to pbgc.gov, we may owe you or your family member's money. marina fox: one quick thing to that, i think it was last year, yahoo! ... to your point that nobody knows pbgc ... yahoo! picked up an article, there was basically a news article about i think unpaid money for pension or something like that ...
anne: we had millions just waiting for you to come claim it. marina fox: so yahoo! picked up that article and pbgc exploded. so don't ever assume, "well, my site is niche and nobody is ever going to see. " if it gets picked up by one of the major news portals, it will explode. make sure you're aware of that and you plan accordingly so your site doesn't go down. that's something to keep in mind as well. male: i was going to say google search alert is one of my favorite sources of analytics for figuring that out.
marina fox: yeah. all right, cool.so i want to take a segue and talk about mobile apps. i think that whole area is a little ... it's a fascinating area because i think it's still fairly new in the government, especially with respect to mobile apps analytics. david is here and he ... obviously that's your background and i'd really like for you to share a story of how you've been able to leverage mobile analytics.
to just going from looking at the numbers, to figuring out the why aspect and drive change. david: thank you. yeah it is still very new within the federal government. i know a lot of you are focused responsive web design and not necessarily mobile apps. there's a couple of lessons that i think we've learned from using mobile apps. one, that what's a good kpi for the private sector isn't a good kpi for us. a lot of the stuff that you're going to find out there on mobile analytics is
geared towards facebook. but for a lot of our apps, they're not necessarily going to be the same kpis. for example, our virtual hope box app is a crisis support tool. a typical kpi is daily average uses, how many times are you looking at my app in a day? for facebook yeah, i want it to be a lot of times during the day. for virtual hope box, i kind of hope you're not using it multiple times a day because you're in crises and you need to call somebody and put down the app.
so, understanding how to translate those kinds of things into what you're doing is really important. a good way to do that is like we said is build a behavioral kind of flow. how do i expect my users to use this app and then tying that back into specific mobile kpis so that you can segment your audience and really see, how many of my users are actually using the app as i intended? then maybe, do i need to add features or remove features?
for our [inaudible] app we expected that people would use the tool as intended to learn [inaudible] breathing, coach them through the process, and then never use it again. female: i have that, that's you? david: that's us. female: hey. david: yay, thank you. female: i use it all the time. david: anyone else?
so what we found is people weren't stopping using it, they kept going and the liked the coaching tool that kind of coached them through every now and again. so with us, it got us thinking do we need a separate stand alone product that just kind of acts as a coaching breathing support tool. and what user features can we incorporate in that because we found a feature that was more popular. the other thing about mobile analytics in the federal government is that mobile
analytics are here, they're just not evenly distributed across the federal government. so you may find that you have more or less access to certain kinds of mobile analytics. but even with a little bit of mobile analytics you can actually learn a lot about what's going on with your product. for example, if you're on the google play store one of the very basic things you get is something called active android installs.
so you can see how many people actively have your app installed on their device. not just downloaded it, but how many people actively have it on the device as it is right now. with some of our apps we found that even though they had been downloaded several thousands of times only about thirty people had them installed on their phone. so with us we were able to use that to make a decision and say, "okay, are those thirty users worth sustaining this tool? " and we decided not [inaudible] with that app.
so making cost decisions you can even do it with very limited amounts of data. anne: great. i think a lot of what you said can apply to not just mobile apps, but in general: operating systems, browsers. we are noticing digital analytics government-wide we're seeing from just the last twelve months the mobile trend has gone from twenty-two percent to now over thirty percent. so the reach of mobile is winning. but i think it is important to go back to the kpi aspect and say,
"why do we have this app? " and ask some tough questions that's worth sustaining it if there's only a small number of people. but in some cases maybe it is worth it. and that's kind of the decision that has to be made using the different types of data, not just the downloads because that's deceiving, that's your volume. it's almost like page views. david: i was going to say, downloads are the new page views by the way, if you don't know.
this is really interesting so thank you for sharing that. want to take another segue and talk about advanced analytics. that area is really exciting particularly because it is to some it's kind of like a "ooh, i don't know how to do that. " this is like the scariest statistician area. but i think it's really a great area, because that is what allows us to take this to the next level and look through big data and find discoveries from the data and find the patterns that would ultimately help us to make better decisions
and improve customer experience. so sam, could you talk about how you been able to leverage advanced analytics at hhs. sam: i don't know how to do that. anne: okay, don't. sam: no, i think it's just pushing the boundaries beyond the tool set and really sticking to your experiments hypothesis can lead you to other things. one thing we're doing at healthcare.gov, well for healthcare. gov is pushing a lot of digital campaigns trying to push a lot of people to
their website. there's only so much you can measure with the tools and we wanted to measure different variables. so one thing we're doing is looking at collecting extended metadata additional metadata could be a tone of voice, it's marketing style. placing of the call of action, whether there's a call of action. length of the message. you need to buy into the content manager, it's not something that's easily done because there's actual work involved.
but if you get folks to agree to collect additional data, even a spreadsheet, the plan is then afterwards after a certain period of time [inaudible] analysis to see if any of those variables make a difference. maybe they don't, but if they did that could change the way you're writing your content for your digital campaigning. so that's one thing we're doing and i guess that gets into the direction of advanced analytics. i would say that with all the buzzwords around government of big data
and data analytics these alerts maybe focusing first on regulatory data, research data, but they'll eventually get around to your analytics team and say "oh yeah, you know would be needed if we had your data too." so i think right now is the time to start data warehousing if you're not already. so if there's not a solid program [inaudible] where you have all your digital analytics, you have access to all your social media tools, all your web traffic tools, all your survey tools. not just in your office, make sure you have it across your agency,
and you're putting that into a warehouse when it needs to be. like some tools, like twitter can only give you three months of data at a time, so you really want to be warehousing that and collecting that because a year from now someone will come around and be like [inaudible]. we should have you guys data in there. i think there's a little weird disconnect because everyone knows that the overwhelming majority ... the way all agencies connect with the public an overwhelming majority of the
time is through the web. but it's not always the first thing talked about when you're in meetings about how to outreach and grow your user-base. it's not always as interesting but they will get there. so i think now is the time to prepare for that move to big data analytics and start warehousing what you can't. anne: i think to just add to that since now we have the government wide digital analytics program, i mean that's the goldmine for big analytics, for advanced analytics.
think about it, as we continue to collect this data the trends that we collect is basically ... we have the entire, if not most of it, of government web space in one place. that really brings a lot of behavioral data in place, it's you know, why are people so fascinated with weather? that's information for further research. why are we so fascinated with things that are so far outside of our control? another thing, right? and of course it also starts to focus us more on what the government can do
better for the consumers, what kind of transactional support we can provide for them. so there's a lot of behavioral data, there's a lot of other information that can be brought together. i think the great thing about this overall government wide initiative is that it allows agencies to collaborate. it allows them to talk to each other and be like "hey, i have this data. " so we created a forum for it so it's not just about the data it's about collaboration and that's how it all works.
going back to [inaudible] to the mass analytics area, i think that one of the things i've always found useful to remember is that [inaudible] analysis's can be done in excel. it doesn't have to mean you have to invest a huge amount of money into purchasing rp or spss or sass. i mean of course at an enterprise level when you're mining a lot of data that software is useful. but if you're trying to do an experiment, and that's what it starts with, you start with small steps.
otherwise if you try to take a big thing right away, you might choke. start with small steps and use excel. it's there. spss is also available for other smaller projects. do a [factor] analysis. again you think it's rocket science but if you're a data person and you really understand the goals, you can do it. that's one thing i wanted to make sure everyone understood is that we're all analysis these days.
you have to be. we're all in a data driven world so making sure that you're leveraging that data to make the experience of websites better for consumers, is really kind of our duty because that's what we do. just in closing, i wanted to make sure that ... we wanted to highlight some of the areas and the key things that we talked about here. you start with a kpi. when you see a lot of data you get scared, you go back
and say "why am i here in the first place? what am i doing? what am i measuring?" success. what is success? go back to the kpis. talk about what makes your website successful. and specially, their journey's, their cases like [use] cases. if you can't think of one big statement, you know, "my website is successful if ..." and you finish the sentence.
if you can't think of that broad statement, then think of five different scenarios that make your website successful. so start with that, and the becomes sort of feedback into your kpis. then go from there. leverage your data, bring it together, never use one source of data, always use it in conjunction with others. always question yourself, "is this right?" is there anything else you guys would like to add before we close? female: i'll just say the next steps for us is turning our analytics into some
[inaudible] we have some neat things going on with some of our [inaudible] modeling and doing some neat visualization things with it, so i'm looking out for all of you guys to see what you guys are doing in that space and we'll be doing stuff soon too. anne: well i know that the panelists have been gracious to say that if you have an questions after the panel, you're welcome to reach out to them, they'll be here.so you can continue this conversation. thank you so much, thank you for being a great audience, and thank you for the panel.
bernetta reese: and we are live at the 2015 digitalgov citizen services summit here in the expo section. first i want to give a big shout-out to everyone online at twitter who is doing a great job tweeting today. continue tweeting using digitalgov15 as your hashtag. i am here with sally from hhs. sally tell us a little bit about what you're doing at hhs in terms of open and structured content. sally: okay,
hhs right now we're actually [inaudible] really big project what's going on where we've taken a really close look at all of our content and the concept is that we're trying to break it free out of web pages. we're no longer looking at content as [inaudible] we're starting to structure it, store it modularly so that it can actually travel so that we can create the content once and publish it everywhere. just this past weekend we launched our redesign of hhs. gov if you haven't looked yet, go check it out it's really amazing
and a lot of the structured content work has actually been what has informed the redesign of our website. bernetta reese: you guys had a really nice video that i saw on [crosstalk] really cool. sally: that was actually [inaudible] in the background. so a lot of the structured content work is actually what made that redesign possible. we started to look at our content and say instead of just putting these blobs of body content on a webpage let's
start and package the content more thoughtfully so that we have really smart short descriptions so that it's tagged with [inaudible] markup or it's tagged with open [inaudible] markup and twitter card markup so it's basically prepared to travel and be set free. bernetta reese: great so do you want to tell us a little bit more about, maybe one of those processes that you use. you talked about the twitter cards. also tell folks exactly, what would you say is open content?
sally: okay. starting with the second question first, open content is really the concept of breaking your content out of web pages. we're starting to think differently about it, we're not creating webpages we're creating thoughtful little packages of content that can just travel away. the way that we kind of went about it is we created these teams that sat down and looked at our content, we did a huge content inventory and we started to say, you know,
what's the important metadata that can inform this content. for the first time ever we're putting just as much emphasis on the content that appears on our webpage as the content that doesn't appear, all of that really smart rich metadata that sits behind it. it's really informing our content strategy. so some of the process that you mentioned such as the twitter cards, the other features that you used during the redesign, tell us a little bit about that. sally: okay so i know everyone experienced it before where you basically ...
you got this great information that you want to tweet out, [inaudible] share with your network and you go to places on facebook and it has this weird image associated with it that you're just like, "i don't know what that is and the short description doesn't match" [inaudible] spending the time now to have our content creators work with us to make sure we can identify the right image that goes with it, we can write a specific short description that would make that content more shareable when it's shared on facebook.
so people would kind of read that short title and short description and say "hey, i get it, i want to share this. " so we're kind of like prepping it. we're going to avoid that situation where you post something on facebook and it's just not right. that won't happen anymore. bernetta reese: so in terms of resources and folks that you have that are helping you to achieve all these great things, tell us a little bit about what that looks like,
do you have a large staff behind this, or kind of ... what's involved? sally: it's been a small team. we've had folks from ... we wanted to make sure we had all the right folks involved, so we had developers in the room, we had folks [inaudible] in the room because that's really important too we want to make sure that our content models has [inaudible]. we have content creators and our leadership team was involved.
it was really sort of a grassroots effort within our agency. [inaudible] that's been a great resource for us and we've been sharing all our models to that group on [inaudible] so that we're hoping that by sharing these things we're hoping people ... they don't have to go through [inaudible] that we did, they can go off of what we've already done. tell us about some other great things that are happening in terms of open projects. [inaudible] that's going to be coming out from hhs
or from your team in your office. sally: if there's one thing that we're really driving towards especially with this open and structured content project it's the concept of being able to create micro sites on the fly. that's really what we're going for, especially in a field like public health that's really important. we're hoping that by structuring our content appropriately, that we can get to the place where there's a [inaudible] we don't have to
scramble and build a site. all of our content will kind of be prepped, it will be ready, and we can pull together a micro site on the fly. huge cost savings, huge savings in terms of [inaudible] time and all the effort it's going to take for those things. i think we're setting ourselves up right for it. bernetta reese: absolutely and that makes a whole lot of sense. so ten seconds, tell us where folks can go to get more information to see your fabulous
redesign and how they can share it? sally: so if you want to check out the redesign visit hhs.gov. take it for a spin, take a look, and let us know what you think. bernetta reese: all right great. thank you so much sally for joining us today, i hope you been enjoying this summit. as you can see there's a lot folks here, it's a great environment, so thanks again. so we're going to bring over our next speaker.
all right so we have karina here. hi karina, thank you for filling in, it's always great to see you too. thank you so much for filling in with us today. we're going to talk to karina a little bit about the us [public participation] playbook, a little bit about the work she's doing at her agency. so tell us karina, what agency you're here with today and a little bit about your involvement with the playbook. karina: hi everybody, i'm karina [inaudible] chief of new media at the department of state with the
bureau of consular affairs. so we get to think about the way we engage with people online through web, mobile, social, and app development. it's a really wonderful thing. and it's made it so that when we thought about the playbook, we had a lot of ideas and almost didn't know where to start. so first of all i'd like to back up and thank you for not calling me lady justin, cause i am filling in for justin [inaudible] today who is our absolute social
gov rock star. he is the one that brought all of this together. i almost balked when he called me one day and said, "i need your help and i need you to start. " justin had this grand idea of calling on seventy federal agencies to talk about what they do well and how we can [inaudible] that for our communities and how we can give back to our citizens to engage with us. i think that's a little overwhelming for everybody. so he said, "i need you to start out." i was scared.
i know what we've done well in the bureau of consular affairs as far as citizen engagement, direct one-on-one, absolutely not just being a government that [inaudible] but asking people to give us information back in and then making actual policy or tech changes based on what people were telling us. i knew how to do that at my job, i almost felt illiterate in how to tell other people how to do that. bernetta reese: so you were one of the first ones involved. you were also one of our team leads and go-owners.
which one were you actually driving. tell us a little bit how that worked. karina: i drove define the mission, which i think is one of the simplest and hardest things to do because as a federal government we know what we want to do as an agency, but we also need to understand what our customers want from us. so your mission is, how do you meet those needs with multiple stakeholders who are wanting different things.
i think the playbook did a really great job of outlining how you're able to do that. the piece that i sponsor was the very first piece, but what everybody else did . came together as a community and said, "this is how we work" i think was monumental. so in terms of leveraging the playbook and borrowing some of the ideas and the steps in there, what is state doing to be able to do that. what are some of the work and the great projects that you're working on right now over there?
karina: well i think [inaudible] helped shaped our mission to be able to say we want more stakeholder input. we want to know what the public wants from us, not just what we want to give them. that means a lot, it means changing contracts and resources and strategies, and that's tough. but what's great is now we have a federal playbook to point to and say this is where our government and this is where our citizens are [inaudible].
it's actually [inaudible] many different projects that we are working on. bernetta reese: yeah and i can see the same at usda having also worked on the playbook with you all. we're planning to leverage it as well, we're planning to launch a digital hub soon. so lot's a great ideas that's going to be coming out of that playbook and that we're going to borrow and we're going to use in some of our [crosstalk]. karina: i know we're talking today about external,
but we're actually going to launch [inaudible] who is on our team in consular affairs, an internal hub. we're going to take the same idea and teach our counsular sessions at 227 worldwide locations how we do this well and how we can support that. we had a lot of people working on this, a lot of folks from government agencies working on it. tell us a little bit about what that felt like, especially us as women right? being on the forefront
and helping to drive a lot of the great work that government is doing now. tell me a little bit about that. karina: well i'm actually glad that you asked that question. i want to hijack the hashtag "changeagents" today because i want everybody to look around the room that we have today and look at the people that are here and realize that fifty-fifty we are women and men in technology. i was one of the first females integrated [inaudible] and combat [inaudible]. that was definitely not the case and i definitely walked into the room
and was a woman first and a professional later. now with the community of practice that we have and sharing these ideas whether it's through the playbook or coming together in citizen services like this, i walk into the room as the professional and as the expert and it doesn't matter whether i'm a woman or a man. so i would like to say that as change agents in the world we are doing it and we need to continue to promote that. also in technology, citizen engagement,
but also in who [inaudible] to the table. so anything else that you want to share about what's going on at your agency, what's happening around government right now, things you are excited about ... let our viewers know what else is happening out there. karina: well i just heard that myusa.gov launched and i'm really interested if [inaudible] can sign on. i want to know all of the technology, that's [inaudible] so when i leave this interview i'm actually going to go find other people that are working here to find out what they're doing
and how i can use that to make my role in government better. bernetta reese: all right well thank you so much karina for your time today, i know you're enjoying the summit we're all enjoying the summit so thank you so much for stopping by. karina: it was great to see you.you too. thanks. bernetta reese: so we're going to bring over our next speaker [inaudible]. how are you today, if you could kind of tell folks who you are and what you're going to do today.
i know you're with bbg so let's tell them what that is. male: sure. i'm a [inaudible] project coordinator with [inaudible] and i master three different [inaudible] developing mobile applications, coordinating mobile applications [inaudible] for the networks within the bbg [inaudible] which creates great video, audio, and blog articles for a global audience. bernetta reese: what is [inaudible]? male: so that's a great question.
bernetta reese: i know you have a panel later today. male: no, that's a good question because it's good to understand what it is upfront. so in software development world the process method which is becoming very popular these days by which one [inaudible] you are able to respond to user needs or changes in technology pretty quickly, which is why [inaudible] respond very quickly. and the card in the process [inaudible] also have very high [inaudible] so everybody than, it's a win-win all around.
so that's [inaudible]. bernetta reese: great, so in terms of [inaudible] and i know we're using that at usda as well. we're in the middle of a content migration ... content management system migration and we're using the whole process of [agile] to help run that as opposed to the whole traditional ... like you said, it keeps things open, it keeps things flowing. so can you talk a little bit about what we can learn,
what government can learn in terms of best practices? when it comes to specifically more [inaudible] you don't award a contract without any requirements [inaudible] telling the vendor, "hey, we want you to bill the [inaudible]. " by doing that you are making the vendor to make to delivering something but at the same time you're also taking your hands off from being able to make the changes which are a must-have if you're building web applications, or mobile applications user experiences key to making it successful. user experience cannot be designed in one go.
you build something [inaudible] get feedback, make small tweaks, to get it better and better so that then you won't be [inaudible] very confident that it could be a [inaudible]. male: so in order to do all that, you put together [inaudible] you assemble it in with the right skills. [inaudible] technology skills, communication skills and so forth and then the federal staff can tell them based on the current understanding what the prototype and then they can also verify the [inaudible].
that sounds about right and very similar ... pretty much what we're going through right now. so those are some great steps, some great pointers. how long have you been doing this kind of work? male: i've been doing [inaudible] development [inaudible] but i've been [inaudible] for the last six years or so. so yeah, i've done it and [inaudible] last two years i've been in the government space, and it's been a great experience.
[inaudible] earlier, so [inaudible]. bernetta reese: that's right, well that sounds great. anything else you want to share with our viewers today, just in general about project management, about agile, based on your experiences, any take-aways? so [inaudible] as they said, [inaudible] for the project because you're working towards [inaudible] small population sooner and two it gives higher [inaudible] and they're focused on doing the web trail
and getting frustrated with a lot of paperwork [inaudible] for the government agency, it gives the most software bang for the taxpayer dollar. so yeah, that's the way it go. bernetta reese: all right, well great. you're going to have a panel later today so we're looking forward to that. we'll hear a lot more from you than, thank you so much for chatting today and enjoy. male: absolutely. bernetta reese: so we're going to bring over some more folks here momentarily.
we're still here in the expo section. folks are walking around and looking at all the different products and services that we have for today having a great time. the vibe is really great here. so we have marina here now. marina is here to talk with us a little bit about the dap. marina you had a great panel earlier today. can you tell us, what is the dap? i know most folks are familiar with it,
but for those who may not be familiar with it, tell us a little bit about the program. marina fox: digital analytics program is a government wide, federal wide analytics program that collects traffic data from thousands of public basing websites that are government websites. essentially if you go to a major government website the consumers ... chances are we're collecting data on it to continuously improve consumer so we use this data to ensure that they're basically delivering what customers want us to deliver.
we're looking at it from a perspective of design, navigation, search, again, to continuously collaborate past the agencies, the government wide do provide notes and ensure that we're providing the best service to the american people. we have rich resources. bernetta reese: that's right.so what's really been working well with the program? marina fox: a lot of things, but i think to kind of highlight the ... three things really. the first one,
it's kind of like a pulse if you will of what's going on the government? because the public dashboard that we released a couple of months ago has become in a way a [inaudible] of what's happening in the nation because the government really touches so many aspects of people's lives? there's so many websites that cover various services that the government provides that if something were to happen in the country, chances are [crosstalk]. for example just two days ago, [inaudible] recalled a bunch of airbags, it was a national recall.
it's been one of the top active sites across the government because this is something that resonates with everyone. we're all driving cars, we all have airbags, that's important to us. during the tax season, it was all about taxes. if the president is speaking, chances are that traffic is going to [inaudible]. so it's really a pulse check of what's happening in the country. another thing is, besides just being in the moment it's also about ... we can see what consumers wanted [inaudible] transactional data. how is the public interacting with the government sites,
and how can we ensure that we have the services for them? bernetta reese: that's right and that's really great that's what [inaudible] want to hear. what has the response been just in general from agencies would you say? i know at my agency we've always been supportive, we've been on-board since day one and we've helped a little to make sure that you guys can get this program up and running. we have great [inaudible] from our leadership,
we're excited to see our agency the usda in the top ten, in the top twenty everyday, every week. what is kind of the feeling around government and what would you say to agency's out there who maybe haven't bought in just yet and are still waiting to get on the platform? marina fox: i think that the products sort of sells itself. i mean, the fact that the government executive been logging into [inaudible] or you can go to the public dashboard and see what's trending over time, how is mobile reach,
how is particular [inaudible] sets the stage as far as strategies is concerned. how can they support? that's why we have the right infrastructure in place and the right [inaudible] in place. before you had to [inaudible] in fact, i don't even think you can do that anymore because this is the combination of so many stuff. but before to even get you an agency [inaudible] you had to [inaudible] for every component in your organization to [inaudible] together.
now, [inaudible] the leadership can go there themselves and make their own database decision. i think that's why it's so easy. once their there, they're sold. [crosstalk] getting them there, and that's what ... we're trying to do the outreach too to accomplish that. bernetta reese: so what is your resources and staffing look like in terms of supporting the program and helping to make sure we get the word out and get agencies signed up
and doing what they need to do? marina fox: we're a small but agile team. however, what's great about this program is that we're collaborating so much across agencies that agencies are starting to participate in the dap open opportunities program. marina fox: so, in fact all of our trainers are from different agencies, from hhs, and we have other agencies participating and they're actually trainers they dedicate twenty percent of their time to
develop or support this program. and of course we actually starting to adapt experts who are again, experts from various agencies who are more technical, more savvy when it comes to data analytics and they're going to be drawing strategies as well. bernetta reese: well that is awesome, looking forward to see more information about that, haven't heard about that yet so that would be great when you guys share more. maybe also ...
i know we're in the middle of a kind of migration right now, we're [inaudible] so we can do some new things with the program. tell us a little bit about the ua migration that's going on right now, some of the benefits of that and why it's so important for agencies to take advantage of that? marina fox: that program is easy to [inaudible] everybody else in respect to skilled ability platform support, everything. so ua [inaudible] it's basically a better, faster and stronger support for all the analytics, and even down here [inaudible].
it's very important that we are on top of things, [inaudible]. well great. thank you so much marina, i really enjoyed your panel earlier today. we definitely know the importance of the program, we heard megan smith talk about it as well. folks are really excited, folks are watching, so we look forward to hearing more from you and learning more about the program here. we are going to move on to our next speaker.
hi laura, this is laura that we have with us here. laura, tell folks a little bit about who you are, what you're doing, we're going to talk open opportunities of course and your role with that program, so let's start about you and what your goal is with our program. laura: okay, so ... hi i'm laura, i'm from the peace corp and i volunteer for the open opportunities program. so what that is is a program that allows federal employees all across
government to kind of work on digital projects at other agencies outside of the [inaudible] agency. so it can be kind of a small projects like writing a blog post, or even longer term things over a series of [inaudible] five hours a week towards certain projects. the kinds of projects [inaudible] can work on range from, if you're an artist you can design an info-graphic, if you're a techy you can [inaudible] or build something or be a project manager, so it's really something [inaudible].
and it's free and it's easy. you only need to get your supervisor to say, okay, there's no [inaudible] there's no red tape so it's regional. bernetta reese: awesome, so you're on detail to the program from where? laura: yes, i work at the peace corp, i'm a writer and editor there. i don't even [inaudible] work [inaudible] digital space during my day job, but it's something i have [inaudible] and i'm interested in it so i signed up to help lisa nelson kind of [inaudible] update these programs, kind of [inaudible] for her.
we send a weekly newsletter. actually this is how you find us. go on the [inaudible] website if you're listening. it's [inaudible] is the website. you can also go ahead and send us an email at digitalgov@gsa.gov. you can sign up to get weekly newsletter where we say what kind of opportunities are available or free for the [inaudible]. bernetta reese: that's right, and i'm also a part of the program on [detail]
and i know we're going to flip this in just a second, i'm going to let you ask me some questions so viewers can learn more about what it is that i'm doing as well. but let's talk a little bit more about the agencies who are involved right now, the number of agencies, the number of open opportunities that you have going on right now and some of the new ones that are coming. laura: i think even more than sixty agencies are currently posting tasks, and taking tasks, and there are a lot of other agencies who are placing tasks.
so it's a program that's up and running right now that's growing and [inaudible] online and you can feel free to [inaudible] or if you need a hand on a project that you're working on, it's also a great space to kind of [inaudible] who else is on government who's also interested. bernetta reese: awesome.tasks normally take about how long to complete would you say? laura: you can do ... there's a lot of tasks that takes actually, just about a couple of hours of time.
you go to something, you write up a blog post maybe. or you research something and come up with a list of tips about [inaudible] digital. or there are things where you can sign upand actually [inaudible] project for the white house, or [crosstalk] so you need to tell us a little bit about that. bernetta reese: sure, so i'm on detail right now, part-time detail to the white house office of science and technology policy. i am working with a small team in leading up the effort there to build a citizens [inaudible] and crowd sourcing toolkit for the federal government.
we have a small team ... which you also came in through the open opportunities program. so it's really exciting, a lot of stuff happened. laura: yeah, great. what do you like about the role, that [inaudible]. bernetta reese: it's a lot of project management, which is also what i do at my current agency, so it kind of runs side by side which is excellent, which makes it really easy for me to kind of fit in.
but i have really great support from my leadership, from my supervisors who have allowed me to [inaudible] this opportunity, and [inaudible] opportunities, so i'm really really enjoying it. we actually have a really large team, outside of my team, that's all working to develop content and ideas for the toolkit. the team that i'm running, right now we're doing an excellent job in actually building it and putting [inaudible] designs around all the different concepts and ideas. laura: that's great.
what would you say to somebody who's kind of thinking about going on the site and seeing what's there? bernetta reese: i would say, maybe talk to your agency, talk to your boss, communicate to them that maybe you need some growth, you're looking for some additional professional development. definitely look for opportunities that would tie back in to your agency and can enhance the work you're doing there. it may not be fully within your position description, but as long as you can bring those skills
and that experience back to your agency to support them in what they're doing. foster those really good working relationships so that your boss knows what it is that you're aiming to do. maybe start small with some of the smaller tasks, you know? then when you're ready jump out there and do some really big ones. it was really awesome for me to actually see the white house when they posted the opportunities, kind of the first time i had seen an agency really post a detail there as opposed to some of the tasks that we were accustomed to see.
so that's what really really captured my interest there. so i'm expecting that you guys maybe doing more of that down the road, or ... laura: yeah, we're definitely going to expand and grow and encourage other agencies to kind of get involved, think about what kind of projects you're working on that would be great and [inaudible] outside, some outside noise on it [inaudible]. it's a really great way to connect with other people. part of my tasks is to interview other people who are working on the opportunities [inaudible] not just here, in the dc area,
so it's been great to just kind of meet other people. bernetta reese: all right, well thank you so much laura, we loved hearing about it. you too. so we really appreciate you all tuning in today and sharing your thoughts with us. we're having a great turn-out, great speakers, we're following you on twitter still so keep those tweets coming in. we're going to pull over one more speaker.
can you tell us your name, your agency, and what you are doing here with us today. israel: sure. i'm israel.we met [crosstalk] ... female: yes we did ... israel: i work for us small business administration as the chief technologists and [inaudible] residence for the office investment animation where i help with the policy and [inaudible] oversight, for [inaudible] to lead the key efforts on [spir sttr] programs.
and spir is not [inaudible] for the small business [inaudible] program [inaudible] finance and capital for high tech [inaudible]. companies who have [inaudible] it goes on and on, the industry [inaudible] to see finance [inaudible] spr program back in the early 90s. seventy percent of the components in [inaudible] are [inaudible] financed from the spr program many many years ago. so spa helps with the program [inaudible] department of defense, nasa, national science foundation, [inaudible], epa.
national science foundation, your agency, usda, epa, education [inaudible], it goes on and on, and i each rung up, so we just redesigned the sbir. gov website so any entrepreneur or innovative researcher needs to know what we do that is high-tech, high-profile partnership, wrong. it's a synchronized platform that connects all government agencies to a centralized site, so any news that goes at spif.gov, you can find funding opportunities across government agencies, [inaudible] and this is just a shot of ? okay.
i'm rolling, guys. you could access three of them and whatnot. we have funding opportunities that are across agencies, so if we're going to do high-end polymer development or you'll do a high-end robotics, you can find multiple opportunities at usda [inaudible] or whatnot. they all lead to different forms of funding opportunities [inaudible] speaker 2: it sounds like, maybe, the redesign has made it easier for folks to find those opportunities ?
anything else you want to say about that, about the redesign and? rob: we used the operating on the [inaudible] so that they too can get to what it is they're looking for. [inaudible] there's a very sophisticated [inaudible] and so whatever words you use or type into, you can look in immediately the fundaments of why it's [inaudible] whatnot, all that stuff [inaudible] then, there's a ton of resources on there for domestic and international perspective on how to commercialize
and scale up the company with resources at the state, both in the local state, federal and individual. speaker 2: all right, so great. thank you so much for stopping by and talking to us. i hope you're enjoying the summit. rob: i am enjoying it very well. speaker 2: awesome, great. it's fun.thank you. we're going to bring over another speaker here.
howdy. we just talked recently, generally. that's right. thank you for coming over to talk to us. tell us your name again, the agency you're with speaker 4: okay. my name is [inaudible] i work at gsa, and the program i work in is called, "sites," and we basically help federal agencies build up their website. that's why i'm doing this, so i have to be very, very careful.
speaker 2: love it. speaker 4: what we provide is an open-source cms, just a word press, and then we take care of all the infrastructure, the security. we help you launch and host the site for you, so if you have a concept, you have a code, worldwide initiative that is open to the public, and we are a possibility for people. speaker 2: excellent. what are some of the projects that are currently, and some of the agencies who are already using this service
and the platform that you have? speaker 4: well, we have several agencies, and we host the department of labor blog. we host fedramp. you know fedramp the program? right now, we have about 40 different sites that are either live or in development, so we're keeping busy that we help to find other people that we can help. speaker 2: that's right,
and we talked about the playbook earlier when korrina was with us. the playbook is also hosted on sites.usa.gov. speaker 4: exactly. i think that it's in ? first, in the case of the playbook is something that they wanted to do and they wanted to do fast. we have agencies, especially after deployment and you're ready to kids doing the requirements and all that, it's really, if you're ready and you have your concept, you can launch as fast as in a week,
almost. speaker 2: in addition to ? i know you guys are pitching one or more agencies to be involved. why would you say someone should sign up and use your platform versus another platform that's offered? what are some of the benefits that are offered in the service that you can provide that others can't? speaker 4: let me tell you. it's not a one-fit-all solution.
there are people who have very special requirements or that need a host of pii. we don't do that, so it's not for everybody. i'm not saying it's a better solution than others. i'm just saying it's a solution that is good for the people who need a public posting website and want to host it, or maybe they'll have the infrastructure or the support, and we can provide that all to you. speaker 2: all right, well, thank you so much. that sounds great.
thanks for coming over to talk to us. are you having a great time? it's great meeting you too. we're getting ready to wrap this up and turn it back over to the main conference room. it's been a great pleasure hosting you all today, and thank you all for tuning in online. please keep including this in digitalgov 15 and keep enjoying. keep watching.
speaker 5: all right. we're getting ready to get started. atencion por favor. speaker 6: louder. speaker 5: we're back. speaker 7: that also works. speaker 5: yeah, just scream like a crazy man. speaker 7: thank you, everyone. thanks to those of you who were sticking around on the live stream.
just to clarify, you didn't miss the first part of that presentation and the last part of the live stream, that was not bob "the builder. " that was cokie, from site.usa.gov, which is a great product we offer here at gsa, but i'd like to take a moment right now to have our summit sounding board stand up. please stand up if you're a member of the summit sounding board. i think some of them are outside, talking, so there is more than the two people who are standing up now, or three.
one of the things we've done is we've been really trying to apply an open method to arranging the summit today. they had great ideas. they've had great input on the ideas you've submitted with the crowd hall, and many of the presentations from this afternoon are taken directly from the crowd hall. when we were talking with the sounding board, we had a couple of get-togethers. we just had them sit around and talk about some ideas. the micromanaging session that just happened came out of that,
and one of the cool panels that i'm excited that's just happening now is one about privacy and identity management in the open, and that really was the brain child of dan morgan, who's the chief data officer at department of transportation. what we decided to do was ask him to lead a panel about: how can government become a leader in privacy and identity management? we know that david bright was talking about the importance of big data. mine was talking about it earlier, and it's really concerned with analytics, so i'd like to ask dan to come up, and he has a colleague from mist
and a colleague from connect.gov to come in, tell us a little bit what we need to do to get prepared and become a leader in the federal government in privacy and identity management. dan, come on up. dan: well, i don't have to say if i'm on picture. come on up, jen. come on, sean join me up here. jen: well, make this by yourself.
dan: in the sounding board session, we had an interesting conversation around security and privacy, and conversations always tend to circle around this notion that cyber security and privacy can be barriers to innovation, and i just vehemently disagree with that proposition. if you're innovative, it includes cyber security and privacy considerations. you have to consider these things, and we need to make it easier to work on this stuff. jen and sean are doing some really great things,
and i want to give them the opportunity to introduce themselves and talk about what they're working on, and then we'll ask her a couple of questions, so jen? jen: right.thank you so much. my name is jennifer cuthbert, and i am the program manager for the connect. gov program at gsa. when we talk about connect.gov, we say it is a secure privacy-enhancing service that connects users to government using government-approved third-party digital credentials.
does anybody understand what that is? all right, so let me explain it to you. right now, when you go to government, you have to create a unique user id and password for every agency you go to, and the one i create at irs is not recognized when i applied for a student loan, but what if i have the ability to bring a credential i trust and that, maybe, i use in the commercial market to government? i don't have to create a government-unique id, and i can choose what credential i use for my interactions on government,
and that's exactly what we're doing with connect.gov. we're using these, what we're calling, "government-approved third-party credentials" so they meet security and privacy standards and then you'll be able to use it to log into government services, so i can use my verizon id. i could use google. i could use paypal, or maybe, my bank. maybe one day, i can use my bank to log in, so these are what we're looking at,
but part of the value, too, with connect.gov, and as you said, we've really thought about the privacy and security, and we built in privacy and security around it. i think you know, right now, if you use your google id and log in somewhere, when you return, they're going to show you the shoes you just looked at, because they're tracking you. they're keeping that marketing data, but if you use google through the connect. gov hub, we prevent that. they don't know where in government you go, and actually,
the agencies don't know what credential provider you use. they just know that you are who you say you are so they can provide you a service, so really, we are the infrastructure to try and enable digital government services, and we've done it by trying to build in the privacy and security, and i think we'll talk a little bit about that. sean: i'm sean brooks. i'm a privacy engineer at mist. i work in the national strategy for trust
and identities in cyberspace program office. nstic was a strategy released by the white house in 2011, and it's designed to deal with the exact challenge that jen just mentioned but not only within government but beyond this issue of having not necessarily the most privacy or security enhancing identity services available online, not having them be interoperable, not necessarily having them be user-friendly, and so in the nstic program office, we have a number of different work streams that we use to contribute to trying to push the identity ecosystem, as we call it, forward
and to alignment with the guiding principles of privacy, security, interoperability and user-friendliness. the work that we do is funding pilot programs in the private sector and in government to find new experimental ways to tackle these challenges. we support a non-governmental steering group to help figure out what the new identity ecosystem should look like if it's aligned to the nstic. we also work very closely with other areas of mist, areas like the development of the cyber security framework for critical infrastructure, working groups on cloud security and cyber physical systems,
a number of other programs to figure out what the identity issues are as core components to the problems that those groups are trying to tackle and figure out how to collectively bring resources to bear to solve some of those issues. one of the most compelling issues that we've been working on is the challenges related to privacy engineering, or as it's been called and its previous iterations in the privacy space, privacy by design, but at the core of it is: how do you improve communications between folks responsible for building privacy into information systems
and those responsible for setting some of the policy or legal requirements that the organization must adhere to, and so we're in the process of doing a lot of work on trying to figure out processes and risk models that are more aligned with the way organizations are actually productive as opposed to, as was just mentioned, some of the more stagnant practices that you see in a lot of organizations within government and outside of government, which makes privacy seem to be a barrier to innovation, new services,
and the like. dan: yeah. one of the things that we see, sometimes, is it's pretty hard for our people who are designing digital services to describe privacy risk or understand privacy risks. the language doesn't work quite yet, right? we've had privacy by design for a while, but it's still very techy and not user-centric. i think connect.
gov is also thinking about how to actually be user-centric in the way that we provision identity services and those kinds of things. can you talk a little bit about that user-phasing bit a little bit more and give an idea of the user experience things that we need to think about as we move this stuff forward? jen: yeah. i think that's been one of the most interesting things as we've designed the connect.gov and working with our partners is the user experience, and how do we come up with a common lexicon, because i can stand up here
and talk to you about levels of assurance or government-approved third-party digital credentials, but if you're a consumer and a user, you don't really care. you want convenience, and you want trust, right? how do i give that to you in a manner that you understand without ruling out all the government speak and making sure you've complied with this special publication 800-63, or whatever, and on the user interface side too, we often see ? sometimes, to try and drive traffic, we default to convenience,
and maybe we don't think about the security or the privacy, and so we are working to try and build the convenience in while still having the security and the privacy, and we've been working with some of our stakeholders that have a diverse population, so veterans is one of them. you have young, tech-savvy veterans, right? they're online. they're mobile. they get it.
i have old veterans that don't even have a cellphone that can't even understand you have disabled veterans that may not be able to use a mobile device, so how do we ensure that we can deliver digital service to everybody in a way that talks to them and works with them? we're doing a bunch of great user testing and research, and this is in the early stages, so we're going to learn a lot, but we're also partnered with a bunch of our international countries that are doing the same thing. we're in a lot of really great user research,
and the one i love to mention is that we often say that we need to authenticate your identity so that we make sure you are who you say you are. people don't want the government to make sure they are who they say they are, but if i say, "we want to authenticate your identity to make sure nobody is pretending to be you," you're all over that, so it's a simple change, but it's user research that helps us do it. sean: one of the things that we're trying to focus on a lot is helping organizations understand what the actual risks related to privacy are that
they're trying to mitigate with controls in their information system. user centricity is something that been a big conversation in privacy for a long time, but it's also often very tempting for that to be a foil for somewhat lazy decision about privacy, because in reality, writing a really long, complex privacy policy is an excellent way to push all the decisions about privacy onto the user, because the organization doesn't have to do anything. they can just say, "here's all the information you need to know in a language that you may
or may not understand, and you can make the choice whether or not you want to engage with the service, and therefore, you are informed. that doesn't necessarily mitigate any actual risks that you might have to individuals when they engage with their system, so a lot of what we're trying to do is help organizations understand what those risks are so that they can build controls whether or not they're user-centric, and to say this actually addresses and mitigates the risks that we're concerned about. one of the things that i like to say when talking about this work is that if i
could eliminate the word, "creepy" from all future conversations about privacy, i would, because privacy is full of these really big, nebulous terms where people say, "things are creepy," or, "they make me uncomfortable," and that doesn't mean anything. that doesn't address any particular problems that folks are trying to deal with, and so when we get down to those actual problems, then you can actually make controls and systems that address? dan: that's great, so how do we help our web and mobile communities learn more about these kinds of issues?
i know you're working on your privacy engineering framework, and connect. gov is still in the very early stages, but where could people go for resources, and what are you guys thinking about in terms of informing the community like the digital and mobile community? jean: i think there's a lot of great resources both in an outside of government. i think, even, we have a mobile lab here. you're hopefully going to talk about some things that you're doing. i sometimes leverage the international association of privacy professionals, iapp, a lot of great resources there, where the key, i think,
to all of this is you shouldn't have to be a privacy or security expert. you should be able to leverage stuff that's already built for you. dan: this is a good time to have this conversation, because we've been doing a lot of work over the last year and a half on this privacy engineering work and the risk framework that we're trying to develop for that. we will have a draft publication coming out that is the summation of our work today, imminently very soon. keep your eyes out for it.
find me afterwards if you'd like to learn more about it, but that would be going out for public comment very soon, and it's going to be absolutely critical that we get comments from federal agencies and not just from the chief privacy officer components of agencies but comments from people who are actually trying to build stuff and do things, because one of the goals of this privacy engineering framework is to make the communication between and across organizations from the legal and policy compliance folks all the way down to the engineering
and development course staff, and even up to ? or executive staff who are trying to deal with risks and make decisions about funding. in order to mitigate those risks, we're trying to make that conversation more productive and be a singular conversation. cyber security has come a long way in the last 10 years about unifying the type of conversation about risks across organizations and privacy has really lagged behind.
when that document comes out, it's going to be really important for us to get public comments from federal agencies about whether or not that's what we've created that's something useful for you. we'll have some tools in there, a set of worksheets that are draft but things that we think are going to be helpful in order to help facilitate an iterative approach to this kind of work within your organizations. dan: i think that's key, right?
there's a lot of really awesome research and work that's happening through this mist program, and it is designed for this community to understand. we could build all the beautiful digital services that we want, but if people don't trust them, they're not going to use them, and if they have to use them in order to do business with us, we would like to tamp down their fear and concern so that we can actually get them involved and be able to use them. it's very important that we address these user experience things early on
and make sure that the people who are building these services understand what we're trying to do and how they can best address these risks. i'm going to leave it with: what's the number one thing that you think we can do to increase our success rate, meeting our privacy in cyber security obligations in digital and mobile? jen: i think we have to move away from the check-the-box, put up the five-page, 2,000-word count privacy thing to make the lawyers happy and really built it in, build that key ? leverage existing technologies that are out there that help you solve a problem.
if you can use a connect.gov service to deliver you the identity, then you don't have to be an expert on that, and you can quickly and easily enable that killer digital application. how do we make it easy and allow you to conform with the privacy and security by building it in? sean: there is so much amazing potential for government agencies to provide new and awesome services using what is essentially sensitive information about people, and that's not just traditional pii, but that's a variety of different information.
we don't even necessarily know whether or not that's sensitive or not, and there's a lot of fear about the type of privacy concerns that can be raised because of the trusted relationship between agencies and citizens, and i think that's it's really important to understand an onboard process within the organization that helps you identify what those risks are that you are actually concerned about, because that's understanding what the actual things that you're trying to prevent are, what the actual problems that might exist for individuals are as you design
your service. that's how you get the information you need to move from, "this is too scary. we can't do this," to "this is how we protect citizens and deliver them the services that they demand in the modern age. dan: awesome. i just want to thank our panelists, and let's move forward. speaker 5: thank you, sean, jen and dan. up next on our agenda is 3d printing, and there's been a number of different projects over the last year about 3d
printing and the federal government. one was at nasa. there have been a couple others, but today, we're really going to be talking about enabling open science the 3d printing, and we have dr. daryl birch from nih to come and tell us a little bit about what they're doing with enabling 3d printing. birch, take it away.dr. hart: hello ? dr.
daryl birch. now, at the nih, we have a lot of doctors. we have a lot of physicians, but normally, we call each other by our first names. i'm happy to be here. thanks for having me. i'm hoping to tell you a little bit about how 3d printing is being used at the nih, and in fact, how we are then enabling that technology across or throughout the world, actually, through a website and a tool,
a collection of tools and data called the "nih 3d print exchange." let's see. as you know, from the state of the union addresses in 2013 and 2014, this is on the president's radar. he said that 3d printing has the potential to revolutionize the way we make almost everything, and this is, in fact, what's happening at the nih. we are creating new lab apparatus with 3d printers that you cannot buy we are using it to make models of biological materials and biological pieces of cells and proteins that allow us to develop new flu vaccines
or new assays for drug discovery. let me go back to one other thought about this. there's a wonderful book by hod lipson, and melba currman from cornell university [audio distortion] "fabricated," and in this book, they indicate 10 reasons why 3d printing is disruptive, disruptive to the way we manufacture, and that's why it's on the president's agenda, because he calls it, "manufacturing 2. 0" that it's going to change the way that we have to do education,
the way that we have to build up factories and the way that this enables, almost, organic farming and sustainable farming in communities, enables better nutrition and better health in those communities. 3d printing does the same thing for manufacturing in communities. it's interesting. enabling open science and 3d printing, this is it, the nih 3d print exchange. this is a screenshot of the homepage here, 3dprint.nih.gov. this website is, as i said, a collection of data, of 3d-printable objects in biomedical science.
it's a database. it's a website that's also this collection of tools, a collection of tools, and all of these things are actually enabled by open-source software and open data, so i have just a few points i wanted to get across to you at the very beginning so that my time doesn't run out.the 3d print exchange depends on api's that we developed and that we also leverage from other people, professional companies, to increase function and make open our data. also,
the software that we've written that glues together all the other open-source software that we're leveraging is also open, and it's on drupal and github. that open source software, it runs the website. it runs the database, and it runs the tools engine. another thing that's great about the 3d print exchange is that although we can create our own data at the nih and we are offering it to educators, to researchers, to whoever might be interested in this kind of biomedical data, 3d data. it also allows crowd sourcing.
in other words, this is a government-sponsored website, 3dprint.nih. gov that has content generated by users from across the world. people upload their designs. people upload their models, their data, whatever it might be. this has the effect of by leveraging the tools that we've built to democratize science, to enable citizen science or diy bio, if you may have heard of those terms before. it also encourages alternative learning. i know,
for example that there's a high school just across the river near dallas airport that?it's a private school?and half of their curriculum is 3d printing, and it really is enabled those students who are failing, actually, in traditional schools to excel and become the tech leaders for tomorrow. another point is that the 3d print exchange is leveraging data that's found in scientific repositories that have already been paid for through government grants, and then allows that data to become more in the hands of other researchers who would find it difficult to access those data repos directly,
or educators for the same thing, kind of the same way that the noaa provides weather data to all the weather companies. all right, so just very quickly, 3d printing, what is it? you have seen some of this in the movies or on tv. it doesn't work quite as fast and well as it does in the movies and tv, but essentially, you design a digital object in 3d, and then lay down layers of material to build up your 3d object, and this is used in a variety of things.
it's hard to see with the chairs here, but there are people using this for prosthetics and other people using this for heart models. in fact, i have one of those here with me. this is a model of a heart split up into pieces that's used in pediatric hospitals to better understand the congenital deformities inside that allow the surgeon to have a better result and a faster surgery just by having this model in their hands. it's a lot faster to look at this and feel it and look inside
and understand it even with, maybe, using 3d glasses or turning things around in the screen. having it in your hands teach us quite a bit. let me just tell a quick story. at the nih ? i'm not sure if this was a movie. it looks like it's not going to work. this is called hemoglobin, which is a protein receptor on the outside of the flu virus, and the researchers have been looking at this for 15 years,
didn't a particular set of mutations, but by making the 3d print and putting it in their hands just as with ? they found how those mutations were connected, and therefore this may lead to a universal flu vaccine. in addition, we use 3d printing, as i said, for those lab gadgets or tools that you just can't buy, otherwise. the 3d print exchange enables all that and shares that out. i'd like to just point that part of the committee who are?what was it called, the citizen?
speaker 5: the sounding board. daryl: sounding board. the sounding board, there are two members of the sounding board, david hale, who's back here, and reed holman, who may be here as well. reed is actually the administrator of this program, hhs idea lab, or one of the folks involved with it who helped us really bring our idea using lean startup methodologies and agile software development techniques to bring this idea from an idea to actually debuting at the white house,
at the maker fair last june in less than 11 months. in that time, and remember, this is kind of a nerdy science website. we have something like 2,000 registered users, 5,000 models and tens of thousands of hits, maybe 30,000, 40,000 hits by now. here we are at the white house with dr. collins, the director of the nih. just one thought i'd like to leave with you in terms of how we've built up this resource is one of our developers on our team in investigating and trying to build the website, he said, "you guys are breaking all the rules.
you're not supposed to do things this way," and yet it was only because we did those things that we were able to have the success and the speed at which we were able to develop the 3d print exchange. i want to see if there's anything else that i want to just say. this is part of the toolset for creating 3d models from scientist data, and what we get out are totally new representations of the data that regular lab scientists would have a hard time creating on their own and then putting that into a 3d-printable format. again, this is some of the open-source software that we leverage
and most importantly, again, is the development of our api's, and we're really grateful to 18f, actually, in great brooks, who's helped champion the development of our api, our adaptive programming interface that allows computers to talk to computers and accelerate the rate of change. we've had a great team working on this, and importantly, also, a lot of friends and advisers from all over the country at scripts and various institutes. we've got the military involved.
we've got the smithsonian involved, so it's really a great collection of people who helped us build up the nih 3d print exchange, including our executive leadership at the nih. thanks for letting me speak to you today, and i'd be happy to talk to you as we go through. thanks again. speaker 5: thank you, daryl. that's a very complex talk that you talked about in 10 to 15 minutes. we appreciate that, and also, thanks for bringing your heart.
the actual next panel has? daryl: don't forget my viruses. speaker 5: yes. he brought some viruses too, so i'll have a cold tomorrow. i'd like to introduce our next panel who also has their own props, which they left in the staging area, so it and digital services, no longer being the team of no versus the team of yes, moving away from being between two firms, which is really what we're trying to do here, and general stevens,
who is the usda cio at fsis will talk a little bit about this, but we're really talking about being creative and talk about all the technology and digital services you want, but you need your procurement, your legal and your security and other enablers in those areas to help you get it done. janet, why don't you take it away with the wolf pack here? janet: just in case we need them. these are up. i think they invited me here,
because i think for maybe the better part of a decade, i've complained that? we all complain?about working across the it or the legal or all the other areas, but we never actually make progress and that we really like to complain about each other. i think that's why they brought us up here today. i wanted to start off with something i read maybe about a week ago from the deputy federal cto and new chief data scientist, dj patel, and he said those of us who are technologists out there, it's not our job, not just to throw stones and say, "hey, how come you guys can't do it this way?
we have to get in there. we have to embrace the problem. we have to own the problem with them." if we looked over the history of how the government has interacted on these technology issues, we are our best, as a country, when we are having hard discussions, very difficult dialogue. when we go and we have these dialogues and we're sitting at the table working for team america and really figuring out what is the right solution,
that's when we're at our best, and that's why we're here today. i could turn things over to the panel to briefly introduce themselves. bryan: bryan dunbar, i'm the content manager for nasa's homepage. i've been doing that for quite a while. i am the person you go to when you want to be told, "that's just a bad idea. don't do it that way." maria: my name is maria hichika. i'm with the bureau of alcohol tobacco firearms and explosives. i am a project manager over there.
gabe: my name is gabe sol. i'm with the department of energy. i'm a procurement law attorney. often, when you see me, i'm the guy that says, "you did, what? " but i hope to be the guy that says, "i can help you do that." kimberly: hi, i'm kimberly honeys. i'm with the united states department of agriculture and the division director for the oversight and compliance team, and i'm in bryan's lane, "you want to go live, when?" and "no,
you can't do that." janet: great. first, i wanted to ask the panel, how do you see yourselves, and how do you feel others see you? do you have any favorite anecdotes? bryan: yeah. in some ways, it's a compliment. we feel like we are, in some cases ? well, we're the public face of nasa on the web.
we feel like we are the people that make nasa look good, sometimes to our detriment, because why can't the cio take nine weeks of the transition period to the new contract to negotiate the task order for the transition and leave us with five weeks to build a drupal cms for then thousand pieces of content distributed workforce of 300 people? that's all you need, right? maria: maria, at atf, we introduced an email in the cloud service, and we had our security and our procurement folks saying, "well,
we've never done that. no, we don't do that. we're special. no, that's not for us," and so we had to get over the "we've never done that" attitude to say, "you know what? how do we get it done?" we sat with security. we sat with procurement. we said, "okay.
this has never been done, but we're going to get it done," and we had some hard conversations, spent many months, but we got through it and how to work the various different language to make the language jive with the lingo and the jargon to make sure everybody understood what we were doing, and we're expanding every day with our cloud email service, and so that is one success story we've had. gabe: as the attorney on the panel, i think that often, my role, at least, if not me, personally, is perceived as a team of no theme comes to mind,
but it's not directly on point. i'm he who is to be avoided until the very end. i think that the success stories, and i'm sure we'll get into more a little bit later, the success stories are when my group is brought in earlier. that's when we can be the most effective and we can engage or at least suggest ways to engage with the text side, trying to accomplish something really cool early on and ways that may even open up possibilities and not limit you,
the innovators to one particular set of technologies that you've been introduced to by a salesperson. that's really how we see our role when we're doing it effectively is really assisting you in getting the job done rather than being a checkbox along the way that you avoid until the last moment. kimberly: yeah. i completely agree with gabe. i think in our role at usda, historically, we've been actually told, verbatim, "you're the house of no." i know that with some change in leadership recently,
those walls have been broken down to the point where our security team within the agriculture security operation center at asoc, our goal is to make sure that we're sitting down with the business folks. we're talking to the pm's. we're talking to the legal guys, and we're not bringing all of our techno babble to the table. instead, we're saying, "how can we better help you fulfill your business and mission? we're going to help you do it securely,
and we're going to make sure that all along the way, we don't lose focus on why we're here, and that's to protect the public data. janet: yeah. in my past, i was both a public affairs director. i've been a web services director. now, i'm cio, so i first came out of it as i'm trying to get my website up; we're ready to go; contract's working well; we got the analytics; we built the business case; it's going to go live soon, and then, "what is this cna thing?
no, no, no. we don't need that. we need to go live, because we've marketed that. we need to go live next week, because we've told people," and it really was the team of no and i was [audio distortion] was there. they're keeping us from doing the mission. what's wrong with them? don't they know why we're all here?" and so luckily, they became cio the same area, and then i started to understand kind of, "oh,
there's this other side of things that we have to do that we're supposed to do that there's laws around that we're responsible for." i really?to all of you? it really has developed into that back and forth, but we can certainly do better. in thinking about ? go ahead, bryan? bryan: i was going to say ? so it has come to point ? our office is often, from the communications side, the office of no, "no, you can't do that. you can't turn the nest a little bit to breathe," that sort of thing. it's trying to get out of that, and not the team of yes,
because just everything you shouldn't say yes to everything. some people really do come up with bad ideas, but it's the yes-if, not "no, you can't do that," "yes, you can do that, if you can operate within these other requirements," and we made some progress in that area. maria: if i may add, being in the it project management side, i think a lot of our procurement and our security folks think we're just out for the newest toy, the newest buzzword, the newest item that, "the cloud is the buzzword.
we got to get out there. we got to do mobile," and then it's not just about doing whatever's the hottest topic and throwing out the old. we are trying to meet the demands of the public, the demands of our employees. when we can put it in terms of the real-life scenarios of, "this is going to help our agency, our constituents in this way, and this is why we're doing it," and put it into those real scenarios, i think it helps smooth out those conversations that says, "okay.
now, i understand. how do we do that together so that it's done well and right and at the right legal boundaries and such that we need to abide by?" janet: i know some of the anecdotes we talked about earlier in the week where the team of no, of course, was i think most popular with everybody, "where good ideas go to die," and most recently, "we rain on your cloud," so we're hoping to get past that. in thinking about the last several years of driving digital services and innovation, what common factors happen when things went right,
and what did you learn when things went wrong? bryan: i'm sorry. i'm going to be the master of the obvious. when things went right, we had talked to each other. maria: early. bryan: yeah, early, as we're making a transition from a contract that we've had for 10 years to provide web services to a new contract. i went in and wrote the requirements for: what do we need from the content side?
we need a content management that does x, y, a system that does x, y and z, not just it's fisma-compliant now that the box with this idea thing. the buzzword, of course, was open-source, "please be an open-source cms. " what does that mean for me as a person who publishes that content and getting those requirements into the procurement so that the vendors knew exactly what i was expecting? gabe: i can piggyback on that. the converse of that is also true. if you have advertised this, made it political, anything,
whatever it is that you are doing is really hot and has to go out on tuesday, if you're telling us on monday, i guarantee you it's not going to go out on tuesday. the problem, it isn't the element of surprise that offends me. it's more that i just can't do what i need to do to help you in that span of time, and especially ? the worst case scenario that i've dealt with personally, everything was ready to go, including ? it had been beta-tested on the president's phone, and the president was excited by the project except that the platform that was
used to build the project was obtained on somebody's personal credit card without consideration of any terms and conditions that they theoretically signed the government up to. it took a few months to undo that and redo it properly, and that was more of a situation where the person who was doing this, just it wasn't even a consideration, because the items he was obtaining were actually free. he was never going to hit the thresholds where it was going to be paid service, so he just needed to sign up and get it done.
it was fine, but it wasn't done correctly, and it just took a few months to work through, and eventually, it came out, and it was fine, and i suppose, well-received, although i think at that point i was so exhausted i didn't care whether it was well-received or not, but if we have been involved earlier and the communication lines were open and not avoided, the point is that we could have done it right the first time and been on track to have it released as per the announcements and the excitement
and really ride the wave of the buzz rather than being behind it. kimberly: i can attest to what gabe is saying as well. i know at usda, we too rolled out microsoft office 365, actually pre-fedramp, so we migrated all of our email over to microsoft office 365 and got it through the ana process before fedramp became a requirement. post-fed ramp, our most recent success with the cloud implementation was sales force. believe me when i say that took a long time, a lot of mashing of teeth, and i really could use those at one point,
those boxing gloves for a couple of the conversations that we had early on, not just internally with our own folks but with the cloud service providers themselves. some of the cloud service providers that have had come in, and i will not name names, but there are issues where, "well, why can't we have a pki server located in singapore?" what's the problem with that? once my brain unwound and nodded and i took a moment and hit the pause button and said, "okay.
wait a minute. there's something here. there's communication breakdown. we need to get more folks here that can help me explain why that's a problem and why this is an issue instead of just charting chapter and verse this, and subpart this says, "blah, blah, blah, you have to do that. " it's more than that. it's deeper than that, so it took security to sit, to be invited to the table, and we were grateful for that,
because we don't like that helpless feeling where we can't help the business because they waited to the last minute. it's not coming from a place of anger. it's coming from a place of frustration where we want to help. we're here for the same business submission. we just have different perspectives and different avenues that we have to all protect to make sure that we converge into a single solution that's good for all. that's when things started to go right, finally,
and so sales force recently was our biggest success. we even collaborated with health and human services to get that one accomplished to the point where we have a single system security plan for all of our agencies to use. it's a software. it's a service, and instead of 300 and some controls for our agencies, we have approximately 15 that equates to the business side, less cost, less level of effort and for the security side, i have one system security plan to worry about before continuing as monitoring
instead of 50 different iterations of it, so that was a huge success for us. maria: yeah, and i will add that when things go right, everybody's happy, but when things go wrong, we tend to walk away and say, "well, we're never going to do that again," but i think that when things go wrong, that is our biggest learning opportunity, and i think, sometimes, we forget to learn from our mistakes, and so when we did the whole email migration process, we had some stumbling blocks not just with security and not just with the legal,
but we did have some technical requirements that we haven't thought through well enough. we got through that, and so with our next cloud project, those things that went wrong with our original cloud project, those are the top things that we'll have to talk about, and those are the top things we said, "you know what? we underestimated last time. this next time around, we're going to make sure that those same mistakes don't get repeated.
" i think that's one of the things to take away is when things go wrong, it's not the end of the world. it's a learning opportunity. janet: how do you think we got to what feels like opposing teams? bryan: everybody's got different requirements, and at least ? i work in an organization that has been completely decentralized for its 50-year history in terms of, "well, this is what my manager wants me to do," and in our organization, if you have your manager's authority and some budget,
you can do just about everything. it doesn't matter whether it's communications or it's security or anything else. it's then, "okay, i've done this. my box is checked. my manager's being served. my performance review is great. his performance review is great, and it doesn't mess with anything else the agency is doing." maria: right.
being on the it side, i think some of our technologists think, "well, procurement's never going to understand what we're trying to do, and security's never going to sign off," and there's this attitude of, "they'll never get it, so why bother explaining? we're just going to plow ahead, and we're bringing them along when we need to. i think the pace of technology has contributed to some of that attitude of, "they'll never get it. they're just going to hold us back, and so i think if we can keep in mind that, "yes, technology is moving forward,
but we need to bring along our partners in security and procurement so that they can understand the pace, they can understand our needs and work together toward that. it goes back to that same theme that i think the evolution and the pace of technology has contributed to some of that discomfort and the wall's coming up. gabe: as an attorney who specializes in federal procurement law, i have no concept of why my field would have that reputation. janet: does anyone else want to comment on that comment?
gabe: i'm sure there are some hash tag sarcasms somewhere thrown in there on twitter for me. janet: they know, hash tag. you know the hash tag. gabe: to piggyback on what maria was just saying, though that the critical understanding, i think, as we approach some of these projects is that the technologies and the innovations that this audience is making happen at a speed, at a rate thousands of times faster than the government operates on in daily
basis, let alone changes. perhaps being the change that we talked about in the first session is all part of that, but understanding some of those constraints from your project planning point of view and from your marketing point of view is also critical, so bringing with the understanding that these other teams, these procurement teams, the security teams, the lawyers, we're not trying to stand in your way so much as make sure that what happens isn't delayed.
if you understand that we have these timelines that move a little bit slower than technology, or a lot slower if the case may be, that will help you better budget and better see your time in a more realistic sense. kimberly: those are some great point, all three of you, and i wholeheartedly agree. i've often said that security at usda is like steering the titanic with toothpicks. you don't really realize how far we've gotten until we look back for a moment
and see where we came from. where we came from was the old, i guess, methodology of being that annoying, little kid, the hall monitor that says, "where's your hall pass? " that's what ana historically has been, and i think that's how we got that reputation for being a house of no, a siting chapter and verse where you're violating instead of stopping for a moment, sitting down and going, "okay. i want to help you, business mission.
i want to help you program off this to carry out your mission. how can i help you? all i ask is that you bring me to dinner before dessert comes out so that we can help you with the whole meal." i had to throw in a usda pun there. sorry. janet: we had talked a little bit off the site about: in government, you have a very interesting challenge of the longer you stay, something's going to change. whether it's two, four, eight years, at some point,
whatever you've built up is going to change. my question is: is there hope for us? are we going to get past this someday? bryan: yeah, of course there is. as i was telling jen earlier, our challenge is that we had taken the walls down between, say, communications and cio, and a whole bunch of new people came in and rebuilt a good number of those walls, and we've been working on taking them down.
it comes down to, again?i'm sorry. it's obvious, but it's true?it comes down to telling people, "these are my requirements. what are yours? do they really conflict, or can we figure out how to ? and it's an ongoing process, and we all, at the very top level, far above mine, in a year and a half, things may all change radically, and everybody will wind up doing things differently, and whatever the technology does for this administration is going to be
something different. the people are going to be different, but the progress, the distance that we travel isn't going to change. we may be moving off in a different direction. gabe: if i can jump out of order, that is what we can really do our side of innovation, leave the technology side of innovation to this audience, but if we can break down those communication barriers that despite who is in the upper management chain, it shouldn't matter.
if we are effectively communicating, then we can meet those new technology goals. we can meet those new political goals, whatever they are. janet: if you had known ? you want to finish? if you had known each other two years ago, what questions would you ask each other now? bryan: i hear you're getting in my way. janet: you want to go out, bryan? maria: yeah.
a couple of years ago when we started with our cloud email project that yes, security was getting in our way, and i guess the question i would ask is, "what scares you most about going to the cloud? " because i think there was that house of no, "no, they're going to say no. they're going to say no." "well, why would you say no? what scares you most?" if we were to address that most scary, the worst case scenario, that would be okay, and i think that would have been a more constructive conversation. gabe: being able to not ask, "you did, what?" but rather, "you want to do, what?
" then makes my job more effective. kimberly: i think i would have asked somebody like maria, "what is it exactly that you want to do? please tell me it's not on tuesday and it's monday. " i think if we all would have been able to sit down a couple of years ago and just try to get into each other's heads a little bit and understand and take that time to understand exactly what it is that your mission is, what are your goals, and as far as what keeps me up at night, lots of things, but that's my type-a personality, but really,
my biggest concern is: who has our data? what are they doing with it? oh, my gosh, i know that they're going to come for this vulnerability or that vulnerability, and it's going to be catastrophic to life, limb, everything that we swear to be here for. maria: yeah, and i think when we have those conversations, then we can say, "well, there are ways to mitigate ? there's risks in everything we do. there are things that we need to cross off and we need to do," but when we couldn't say, "honestly,
this is what keeps me up at night," we can talk about, "well, how do we mitigate those risks? what are some things that are there?" and that those things may be addressed and things that we can do about, but if we don't know what those things are, it is not going to happen, and so i think [audio distortion] kimberly: thank you for that.maria: events like this, right? kimberly: i need a moment. maria: yes.
yes. in events like this where people come together and talk to each other and listen to each other and figure, "okay, well, there are success stories out there. there are people who are making it work," and so regardless of administration changes and changes in senior management and things like that, there are people here who are honestly and genuinely interested in getting things done.
janet: i'm asking for closing remarks and a tweet, jump in when you're ready. bryan: i'm not going first. maria: this is a new era, and we are no longer the house of no. come talk to us. gabe: #callthelawyers. kimberly: i don't have a good one. bryan: as i was saying earlier, i use twitter only receive mode, so sorry. kimberly: how about, "nasa is cool."? bryan: well, that goes without saying.
maria: for mine, i have from ideation to implementation, we are all one team, one purpose, one mission, #digitalgov15. speaker 5: fist bumps, fist bumps, fist bumps, fist bumps, all right, so that was a fun one, actually. i did not know how that was going to go, especially when they were talking about bringing boxing gloves yesterday. we've talked a lot about the enabling of great technology, but how can we talk about government changing, the way technology changes the way government works.
as i alluded to, we have the sounding board, but when we had the sounding board, we also had the crowd hall tool. we were soliciting ideas from you all, and you gave us over 90 ideas. two of those ideas were by two guys who were on the same team. when we were buying their votes, they were actually the number two idea on that crowd hall, and they are both from 18f, which i'm sure you've heard a little bit about, and they work right here on the fourth floor in this building, so go and checking out after one o'clock,
but what they're really going to be talking about ? we have john and susean talking about opening government up and transforming regulations to open technologies. john and susean, take it away. susean: okay. speaker 5: okay. susean: hey, i'm susean. this is john. we're from 18f, as you just heard,
and we're here to talk about a tool called eregulations that both of us who were on the team that built it when ? and this was when we were both working at the consumer financial protection bureau. now, we work for 18f right here at gsa, and john's going to take it away. john: hi, so we're going to start by talking about how we build this product from things that we learned from users from the user needs of these regulations, and this all started?this is a piece of a regulation?
this all started when two members of the cfpv reg. division, pam and dan came to us with, really, a request. they didn't have a solution in mind, but they knew the regs team took calls from industry to offer oral guidance, to answer questions, and they were getting a lot of questions that the answers to which were already published in the regulation. the problem is people couldn't figure out how to find it, and couldn't figure out how to get to those things,
so we knew this was a pinpoint for industry, because they were talking about the question. there was a pinpoint for the regs team, because they were spending their time answering questions instead of doing their primary job, which is observing how regulations are doing and revising them. basically, they were in customer service, and they were like, "we know that this can be better, but we don't know how. can you guys help us?
" we started with just a couple weeks' worth discovery where we took that list of regulations online better and broke it down into manorial pieces that we could try to address, because, "make regulations better," that's boiling the ocean; you're not going to be able to make any progress on that. we were able to identify that most users were having discreet issues. they didn't know that that relayed information existed. if you look up here on the screen, there are terms in that that are defined, which you can see here, but if you don't know that they're defined
and know where to look for it, you can't find those definitions. there is interpretative explanation that appears in appendix to the regulation, but you may not know that those exist. regulations change over time, but to see how that works, you wind up having to dive into the federal registrar and see that we are changing paragraph x to read y instead of seeing the changes. we know that people get lost in the regulation. there are some very long sections, very long paragraphs.
it's easy to lose track of where you are and be reading a part of a reg that doesn't actually apply to you. that's important, because it was not always clear what parts of regs were pertinent to what you're doing and which parts you didn't need to address. we also heard that people are keeping notes in outside tools. a lot of people had done this in paper, in books, and they would highlight things. they would keep notes in the margin.
as things went digital, they had no way to do this, so they were keeping notes in excel and word, and things just weren't syncing up for them. we heard complaints, a lot of complaints that just the language is dense and hard to understand. i'm not sure how well you can see on the monitors, but it's a grade 14 reading level for this paragraph of the regulation. we can't always do things about plain language and law, but we can do things to try and make that a little bit easier,
at least with the presentation. we broke these up into these manageable pieces, and basically, we had user needs across the top and then what we did is we structured out things that we might try that we thought would help people engage with regulations better. we structured this out. you intentionally can't see what's on the card, but we structured this out into manageable bites of work, so that as we go would go through, we do one piece.
we would release that piece, get people using it, observe how they're doing, see how we did. if we're doing well, move on to the next piece, if not, take another swing at that. a couple of advantages to doing this is that one, you reduce your risk, because you're doing a little bit at a time, so if something doesn't work, you haven't spent five years developing a big solution, but also, it lets you get some value out for users as you go. every single step of this, every single card on here,
every chunk of work is a valuable thing that people can start using right away instead of waiting for a big solution. we would attach this ? we would worry about one piece at a time and you don't have to design to be bulletproof for the future, and you don't have to worry about what you're going to do in the year and a half. we would take each of these pieces. we would look at what we knew about the users. we would look at what we knew about the business requirements.
we would do a design studio sketching session against just that piece, come up with a bunch of ideas, see which ones we thought might work, the designer on the team, jen, who might be in the room?say hi, jen. she's not here?would sort of take that back, come up with a couple of directions that we might be able to do things with, and just doing a small piece is releasing iteratively, being willing to throw something away that didn't work. there were things that we tried three, four, five times and kept putting in front of the users, and it didn't work right,
and also being willing to, if we get to a solution for a later piece and it requires us to change something that we did earlier, not being afraid to do that. this process really helped us push this forward, and really, the key here that we kept talking to users as we went. we had internal users who were using this tool in beta for their day to day work, helping us find things. we were doing usability testing and interviews with external users, and we just kept going back to the user again
and again to validate what we were doing. now, susean's 01:10:00 going to talk about how we develop this a little. susean: what john has described to you is sort of the process by which we learned what the problems in the space were, and i'm going to take a little bit to describe how we solved some of those problems. basically, what a regulation is, and how many of you have actually read a regulation, or parts of a regulation before?
quite a few hands up, right? well, what a regulation is it's just a block of text, and so to be able to do things with this block of text, you have to figure out what this small unit of computation is, and so if you think about the entire regulation as a unit of computation, well. that's a pretty big piece, so one of the cfpb's regulations, regulation z is actually longer than war and peace, so that's pretty long. that's too big. that's too big a unit of computation.
you can't really do something with that much text if you want to make changes to it. if you think about the smallest unit that makes sense, you're talking about a single word, so we could keep track of the words in the regulation, but guess what? one of the problems with that is, one, it's a lot to keep track of, war and peace, lots of words, but also, at that point you run into this danger of not being to order those correctly. now, you have words out of order, so where do you land up?
well, it turns out regulations are not completely like giant blobs of text. they're actually fairly structured. they have sections, and they have paragraphs, and they have sub-paragraphs, and so this means that your paragraph can be a unit of computation, and this is important, because now, each paragraph, and especially in cfpb's regulations, for example, each paragraph has a unique identifier, and so if you can ? it's really hard to read the screen in here. if you can read the screen, for example, there is paragraph one,
paragraph two on the screen here. those are in section a, which is in section 31, which is in regulation 1005, and what i've described is a way to get to that paragraph, and so if you imagine a tree growing out of the ground and then you pull the tree out of the ground and you flip it around; there's no roots, so you're talking about the trunk and you're talking about different branches, that's how the regulation is structured. that trunk portion is the 1005.
the a is one branch, and then that branches out together. what you're seeing here is a couple of other things, going back from what we learned from users. this is how we solved it. you're seeing links to internal citations. you're seeing defined terms. there's a defined term on the right over there, remittance transfer. you're seeing the clear hierarchy in the text if you've ever read something from the federal register.
you don't see that hierarchy. they don't indent stuff enough, and so how do you build all this? well, you can get someone to type it all up, but what we actually did is we automated this. we didn't want to just do this once. we want to do it again and again and again, so the way to do that is to build software, so we built three pieces of software, actually. we built a parser, and what parser does is it teases out that structure,
that regulation that we talked about, automatically. it figures out where exactly each paragraph is. it builds that tree we've been talking about, and then it takes that and it puts it into a database so that we can serve up those paragraphs. now, we have something that we can act on, so what do we do with that? what we need to do next is we need to pull out citations. we need to pull out terms that were defined in the regulation. we need to pull out external citations, and so we run parsers now not over the original text but over this tree.
we're layering information on top of this regulation, and that's how you end up with a page like this. actually, i'm going to skip this slide and come back to it. what you see on the left here is ? so regulations are interesting in the fact that they're not static. regulations change over time, and in fact, over a 10-year period ending in fiscal year 2014, the federal government made 36,000 changes to regulation, so that's a lot of change happening all over all the time.
what that means is that if you're looking at a particular regulation, it's really important to go back and forth in time. to do that we actually had to read notices, using software, read notices in the federal register, parse those notices, figure out what changes they were talking about, automatically and compile these versions, because guess what? the versions of the regulations aren't easily available. we wrote all the software to do this, and so that's why you get a little ? you can go back and forth in time.
we like to call this "the time travel piece," or maybe i like to call this "the time travel piece. " what i've described here is you see it for one regulation, but what it really is, it's a platform. what we've built is a way to do things. we've taken regulations which are admittedly not very fun for you. we've turned them into data. we turned them into this tree structure, and we made them actionable. now, we can do stuff with these,
and what are the sorts of things we can do with them? one of the things we can do with them is we can now compare different versions and tell you which piece was added and which piece was removed, and if you make thirty six thousand changes a year, this starts to become very important. let's talk about what you can do once you have a platform, and one of the things you can do is you can ask, "which parts of the regulation apply to me?" if you have an faa regulation, the same regulation can apply to the airport and the airline.
well, if you're an airline, you don't actually care about the airport. you just want the airline piece. you can figure that out. we talked about parsing these regulations. well, if we actually turn around and build authoring tools, we eliminate errors, and we can help tease out that structure when it's written, and then the final piece here is that as you're reading war and peace, this federal "war and peace," which is beautifully written, all regulations,
[audio distortion] you want to make notes to yourself, like, "oh my god. this paragraph made me cry," these personal notes, and what you want to do is just like on your kindle, you want to be able to share those notes with others, and that's what you can do with annotations, because you have this tree, you can attach those notes to specific paragraphs so they have the right context. john: maybe you keep those annotations to yourself. maybe you share them with your team, and maybe you share them widely.
it's not a huge leap to get from annotations to a finer-grained formal comment in a notes and comment process, where, maybe, instead of having a comment on a 500-page proposed rule, we can have a comment on then change to the one paragraph that a particular community bank or small business cares about, which helps both the agencies and industry zero in on what needs to work in a reg, and as we start to build up this data and have these people using it, we can start to build visualizations. this is from a different project we did at cfpb around the disclosure form,
but what we did, we allowed people to tie comments to individual pieces of it and they could then ? excuse me, and then we built these [audio distortion] see the parts that people were engaging with and the parts that they weren't. we could do the same thing with long passages of text which would allow regulators to know what they need to zero in on. if they're doing a notes and comment period, we can see that we need to pay a lot of attention to section 25,
because we're getting a ton of comments. if we're outside of a notes and comment period and you see that there are a lot of annotations around a paragraph or a section, maybe you can figure out ? you can look into why, and maybe it turns out that it's really confusing and no one understands it, or maybe it turns out that the regulation has fallen behind the state of the industry and it needs to be updated, or maybe it's something else, but if you can see how people are using it, it lets you understand ? it lets you know where to look so you can understand why things are happening.
maybe more slides, though. susean: clearly, what we've talked about here is regulations, but what we've actually talked about here is a way to do this. we walk you to the process of talking to customers, talking to people that listen to customers on a daily basis, figuring out what those problems are, figuring out how to take something and build a platform so that you can start solving those problems but also teeing it up so that you can solve other problems. one of the important things is we did this ? so we turned this into data.
there's an api, and we did this all as open source, which means that multiple people can work on it. we said we were from 18f. we've been talking about work we did at cfpb. there's a table outside with adam scott on it, who is... john: stand up, adam. susean: ?an awesome person, so talk to him, say hi to adam, and you can get a demo of this tool, and one of the advantages of it being open source is 18f is going to take some
of that source code and work on the notice and comment piece. we have a developer at the federal election commission that's working on getting those fec regs into the tool, and that's the power you get by being open all the way through. speaker 5: thank you very much, that was a great presentation, john, susean. i mentioned that that was number two in the vote giving on our crowd hall back from january and february, when we had you guys voting on ideas, but the one idea that's got the most votes is ? so this is our top vote getter. it is going to be presented by stephanie way,
the director of the innovation lab at opm, and the topic that she said? this is word for word?that you guys all voted for, "how opm is using human center design to transport jobs." stephanie, take it away. stephanie: thanks. hey everybody. my name is stephanie way. can you all hear me? good.
i want to give a shout out to usa jobs, friends and family who are out in the audience. can you all raise your hands, and i see michelle go there early, the program manager, dan thibodaux, assistant, julie, and peter towhee in the back who have all been integral in this project, so i wouldn't be standing here without them, and they're standing here with me right now, so i definitely want to recognize them. yes, i am the director of innovation lab at opm.
have any of you been to the lab at opm? all right. how many of you know or have heard of human center design? even more, all right, i'll take that. well, first of all, so today, we are talking about how we're using human center design and partnering with the usa jobs program team to improve usa jobs itself, that user experience. first i'm going to tell you a bit about the lab
and why we're using human center design in this process. the lab at opm is a human center design based innovation labm and we are charged with building innovation across america's workforce, federal workforce. we are here to partner with ? we may be seated in opm. we actually partner with people across the federal government, agencies, programs, to work with them on the really complex problems and to help them, with them, apply human center design to come up with some innovative solutions to those
complex problems. we do that in three ways: we lead, do, and teach. those are our three goals. by leading, we mean going out to federal agencies, spending time with them to help them understand what innovation means to them. how does innovation work in your organization, currently? what do you want that to look like? let's help you think about how you can get there. we service subject matter experts to the federal community
and also serve as a hub for innovators around the federal government, be, then, human center designers, lean start-uppers, idea hubs, whoever it is, whoever feels like they're innovating, or wants to innovate, we bring them back to the lab to share, very informally and very down to earth, let your hair down, talk about the challenges you're experiencing in innovating in the organization so that we can learn from each other and build a community that crosses the boundaries of agency siloes. doing: our second goal, we actually conduct human center design projects,
as i said, with federal agencies. this usa jobs project is an example where we integrate with the team and do it with them, not for them, so they also not only learn from our subject matter expertise and applying it to their problem, but they also learned human center design themselves so they can then start applying it in their own organization as well. lastly, teaching, which we do in "doing" as well, right, but we actually have a monthly boot camp where people can come
and learn human center design from anyone in the federal agency to learn some of the basic tools, techniques and ethos that they can bring back to their agency. all that being said we then took our human center design approach, partnered with the usa jobs program team to understand how we could improve the user experience people encounter when they're applying it to federal jobs. that's pretty big. how many of you all have actually gone to usa jobs before and tried to apply for a job?
how many of you really love that experience? i saw a half hand. see? it's not all bad, right? snickers, i hear, so there's a lot that can be done around this, and we are really excited to apply human center design. it's really important, and our approach is different in the sense that we actually start with understanding the applicants, the people who are using this tool the most,
and understanding what their perspective is like, their emotional needs, and their challenges with the system are, and we don't just do that in a kind of immediate touch point like, "when do they get to usa jobs? tell us what you don't like about usa jobs. " we actually take an ethnographic approach to step back and understand the entire environment in which usa jobs is operating. the different policies and programs and regulations that are underneath the usa jobs platform that actually
constitutes civil service hiring. we also think about, before they even get to the federal government, we want to understand who they are as people. what's driving them to the federal government? how can we really understand what their needs and experiences are and not only understand their visual cues, and when they read usa jobs, but before they even get there, what are their other online experiences, and what are they really looking for at a deep-seeded net level for usa jobs to fill that platform that and user experience, and we take that
and we move that through a formal methodology that has a lot of informality and room for creativity in it. i'll quickly share some of that today. this is about human center design is developing innovative solution based on the user's needs, not on perceived needs, which happens so many times in the private sector, and anywhere, really, is we think we know our customers, but actually, when you go out and talk to them in a very deep level, we find a lot of aha moments that get to the root causes rather than the
symptoms, and so we understand real needs, not perceived needs. how did we do this through usa jobs? we went out, the collective team went out and did in-depth one on one interviews with people all over the country from different demographics, federal, non-feds, different disciplines, different races, different ages, you name it, to really understand that in-depth context of who they are and what they need and what their experience with usa jobs is from cradle to grave, really,
understanding that comprehensive system, as we said, and you can actually see director archuleta in one of these pictures, where she went out on the road with us as well to do some of these focus groups that we did along the way in addition to one-on-one interviews. then, what do we do with all that data? human center design, as opposed to kind of taking pieces of data and entering it into a spreadsheet and kind of trying to do some sort of quantitative analysis, what we really do is externalize all of our data, because what we can do,
the human brain has the capacity, has a much higher analytical capacity when it sees all of its data in one place rather than trying to search for data through different spreadsheets and clicking, so we took, we had, probably, about 7,000 data points when we finish all of our interviews and we culled it down to a key 2400, so you can see all these post-it notes right now. on each post-it note is a piece of data, either something a user said or something we observed, because a lot of times,
the observation is key in human center design, because people will say they do one thing but in fact will do something entirely different and we try to catch those moments, because that's where the real understanding about behavior and needs in that system and for that user are. we posted those all up around the wall and found meaning from them in a very, maybe, different way than we're used to, traditionally, which is we take data and we analyze it, break it down, try to understand the key components.
we go the opposite way. we take data, and we synthesize it up. how does this data work together to tell us about broader insights that are going on in the system with these applicants, with the hiring managers, with the hr specialist, with the different houses, the different people who own the different system components of the entire application process, so not just usa jobs but once an applicant's leaves usa jobs and goes to another agency to finish submitting their application,
so really understanding all of those different users. what do we see systemically as some of the root causes that have these provocative aha moments that people may not necessarily always want to talk about, but more importantly don't always know what's going on? we really tease those out in this process and we use these insights to then drive the design ideation that comes next, so what do we do? we did code design sessions, actually, with a lot of different users from around the federal government
and with the public. that's another key component of our approach to this process, which is bringing in in-code designing, so we're working with the public to design for the public and that there's this real-one on-one engagement and bringing the federal employees also back in touch with the people that they're serving. we brought together hr specialists and people from around the government and again, the public and private sector, to help provide their ideas.
here's what we're hearing, "what do you think about that? can you help us come up with ideas as well?" and so we did these collaborative ideation sessions to then, based on those insights and those foundational design pillars of what the real needs are in the system, collectively, it really helps drive us toward a place where we can ideate very divergently, build this divergent space and open up room for tremendous creativity so that we actually create room for
innovation in and of itself. we think of really different ideas here that we would never give ourselves, typically, permission to do otherwise, and so we build that space with our code designers and then start to funnel down, "okay, there's a divergent phase of immense creativity," and then we start putting in some realistic constraints into it, budget, time, impact, efficiency, all sort of different parameters, because at the end of the day, we may all want to wave a magic wand
and have something happen and we can talk about that in the divergent phase, doesn't actually mean that's going to translate in the real world, but it gives us room to think of other ideas we wouldn't have thought of otherwise. then, we're going to move ? so this is where we are, right now, in the process with usa jobs. we've actually come through this divergent phase and have culled down to key core components of suggestions for where we think the website can go
and that user experience can change to improve that experience they have online. we've also come up with ideas around those other comprehensive environmental and system changes that recommendations around those components that we don't have sole authority over, certain policies and regulations that aren't owned solely by opm but we think are really important in thinking about changes systemically, again, moving that kind of web of interactions so that it's not just one end that we're pulling but here are all these other ways in which these components and these ideas would influence and change behaviors.
we're going to be making recommendations around that as well, and then this summer, we're going to be moving into this rapid prototyping phase, so instead of trying to build usa jobs into a perfect entity, this redesign of user experience and get it perfect and then release it, the usa jobs program team has actually already implemented this kind of behavior in their program of agile fund development, meaning we don't wait until everything is perfect. we keep trying to just make things better incrementally in as fast as we can.
they go in 12 week cycles to do that and move it out, so we're going to move into that as well. we're first going to start doing just paper prototyping of things, working with as many, again,co-designing with people around the federal government, as well as the public, to get their input into these ideas and improve them really quickly really cheaply before we start building the system, and then continuing to do a lot of testing to make sure we're iterating as much as we can to get it to something as good as we can, but those increments.
there won't be one big rollout. we're moving into that as michelle talks about all the time in usa jobs, this iteration, constant improvement and iterations, and so there will be certain chunks of releases that will have elements of this change in it, but there will be no one big release, and i think that's a really wonderful way of bringing in change into the culture of government and how we consider delivering to the citizens and the people we serve. i'm pretty sure i am at my one-minute mark ? no two minutes.
am i allowed to do questions? can i do questions? no questions, okay. michelle, is there anything else you want to add? it feels weird being across the room from you and you're not up here. great, yeah, so i'll repeat what ? awesome, so there is a usa jobs program table outside, and michelle just invited [audio distortion] i want to be one of the helpers, the prototypes, the co-designers with us.
we're going to be moving into that this summer, and some of the first public releases should be around fall, winter of this coming year, but we're actively looking for testers and co-designers, so if you could stop out at the table and you're interested in contributing or know other people who would, be it your mom, be it your son, whoever it is, we're really excited about that, and also, if you're interested in applying this process, be it human center design or agile fund development, certainly reach out to michelle or me
and we can connect you with the people that will help you think through that. i know there's been a lot of sitting and listening, but i really appreciate your attention and energy and hopefully, we'll see you down in the lab with prototyping. speaker 5: thank you, stephanie, speaking of questions and answers, i know i'd like to know what people on the live stream are eating for lunch, personally, but it is getting ? later in the day, we have two more sessions. each are about 15 minutes, but as folks are leaving, heading to lunch--i know you have busy schedules--in the back of your badge,
in your placard here, there is a survey. we will be surveying everyone after the event, but we really want to get some feedback on your initial reactions to the event, so if you can fill that out before you leave, that will be great. i'll also say that before our customer experience panel, which is coming up next, but right now, we have two gentlemen who i've got to get to know very well. one is bill brantley from usda who has been our digital blogger, who does our weekly api briefing every week with a new api that's either
happening or going to happen in the federal government, and then there is somebody i've just recently learned about, and this is a panel about opening methods for agile development in our government, so we actually found ourselves this firm master who has been working at the broadcasting board of governors. this is [[ashik 01:35:48. ] bill is going to talk to [asik] about some ways that our government can start to adopt agile more and more.
bill and [ashik], come on up. bill: how is everyone doing, having fun? i don't know if i'm agile enough to balance on this thing, so i'll stand. is that all right? so we're just going to kind of start off here since we have a little bit of limited time and he's got so much knowledge to share. we all heard agile. we hear agile as a buzz word. we have agile workforce, we have agile organizations.
my steering is agile. when we talk about agile, what do we actually mean? ashik: a good questions so in the software development board, you collaborate pretty closely between the user community, stakeholders and the development teams. you take the feedback from the user base and adapt yourself, adapt the product that you're building to suit the needs of the users, and also in these days, technology is changing very rapidly, so you are after the technology as well.
all of this makes you agile, or ability to respond quickly to changes and that's what is agile and the definition of agile in the software development world. bill: okay. i remember when i was starting off, we used to call it "rad jad. " anyone know what that was? rapid application development, joint application development, and the whole idea was that you're advised to go and marry your customer, and they're going to be in a room with you
and you're going to build your application with them. that was back in the old days, 1990's. one of the things we want to talk about is agile, and how you want to use it in governments, so exactly, how do we use agile to make government more open and more innovative? ashik: certainly, agile has this slogan of transparency, inspection and adaptation, meaning when you're building something, you don't start with a fixed plan and try to run with the plan. you try to have ? of course, suddenly,
you try to have a vision of the end goal, what you're trying to reach, in this case, for example, provide the citizens a particular service, maybe, but you build a small piece of software and then roll it out to a certain population, monitor the user experience, get feedback, look at how they are using it, where it is working well and where it could be improved upon, where they are struggling, and then try to adapt. transparency, inspection, and adaptation, that's the most agile software development, so it makes, within the team,
for example, everyone can see what everyone else is doing, transparency, open, and again, within leadership and management, they can see exactly what the team is building, where they are, how far they are accomplishing the goals they set out to reach or how much farther to go and whether they are struggling in some particular piece, all of that, much more transparent, and even agile and must come teams favor open layouts, so there is much better opportunity for communication. bill: okay, great.
i think he wants the mic closer to you. ashik: sorry. bill: it's all right. one of the things we were talking, and i was talking at another conference about with agile and trying to introduce in the government, we've been talking about some of the challenges, and people said, "well, i wish agencies had agile budgeting, because that seems to be an issue with agile projects," as many of you all
probably know, so what are some challenges in introducing agile into government, and can you give us an example with successful implementation of an agile project? ashik: sure, i mean i would say having worked for commercial earlier, assistant master, again. i'm currently in the government. commercial has it easy, very easy, because they have a ready measure: money. they put their bets on where the most return on investment is,
but in the government, it is much harder to measure what the value proposition is, much fuzzier and that's a big challenge to start with. second, silicon valley started fairly easy, and they had a company, maybe, not much bigger than this company, five to nine people, but even the smallest federal agency is probably, out of magnitude, much bigger so there are challenges of a large organization and the related issues between, or even the small agency might have thousands of employees,
and many more thousand contractors, so you are to implement something, implement change within a much larger organization, huge challenge by itself, so all of these bring extra challenges in the government space, but you have mentioned budget, so one of the things that ? i am a contractor, but the director there tells me that the way he sends out contracts for agile software development is not based on requirement documents. requirement documents are great. you are binding a contractor to deliver on whatever is agreed upon, but at the same time,
you are also tying your hands on being able to make changes and respond to user feedback or acknowledge changes, so instead, what he does, he sends out on the call orders, bba is asking for specific skillsets needed, so a team has put together based on the skillset we needed, and then he keeps the flexibility of pretty much every two weeks, reprioritizing what needs to be built, and then pretty much every two weeks, looking at completed code delivered and then verifying that that's whatever has been built has been up to his specification.
that's where this is hugely different with the government space. bill: okay, great, and let's say i'm an agency or hearing all about this agile stuff, and we want to introduce them to our agency. how do we actually go about that? how do we learn more about that? how do we tell our bosses what this is and to convince him, make the business case for, "this is why we need to go agile. " how do we do that?
ashik: sure, because agile is such a buzzword these days, there's a lot of material out there. you can learn about agile inexpensively, watching youtube videos or online material. there's plenty available. if you want to do it with more structured fashion, there are the well-established certification organizations. scrum alliance has been around for more than 10 years and well-established and certified by scrum alliance.
more recently, scrum. org is another organization by the two founders of scrum which is the leading process if in agile, so you could do certification if you have a little bit more budget, but more important, like you said, you got to convince the leadership, and the leadership must be sold on it and even though they may not have time to go through all this proper training and so forth, at the very least, they need to understand what it is and then convince themselves to be able to articulate through the entire
organization that "this is the way we are going to move forward, and here is why we are going to use agile and scrum and so forth," so that message is much clearer to the organization and everyone can move forward in a coherent fashion. bill: perfect, perfect, so i think we have some time. let's go for a bonus question here. i'll put you on the spot here. i didn't prepare you beforehand, but what is a scrum project like? what actually happens?
can you give me kind of an idea what happens in a scrum project? ashik: sure, so one thing i do as a scrum master, i mean is who are in typically the federal employee is the person who has got the vision of what they want to build so get that person who has got the vision to share with the entire development team, "what are they trying to accomplish? " it's not based on, "here is the requirement document. this is what it says. i'm going to build it regardless of whether it makes sense or not. " everyone is working towards the business goals, so that's one thing.
then, we do plans for what we are going to build once every two weeks. you could do it every four weeks, or you could do it every one week depending upon the type of project, but two weeks is a default used within the practice. in the two week timeframe, the person with the vision prioritizes what they want to build and then it is cut out into smaller pieces of work to be done and the development team ask questions, get clarifications and they go ahead and build it, and at the end of the two weeks,
they will demo the actual working code, "here is what ? this is the screen comes up, and the user does this, and then this is what happens," and so forth, so every two weeks, you can actually see working code and exchange circumstances. every two weeks, you can actually deploy the code to the end user, and so it should be potentially shippable code, fully tested, ready to roll. bill i think it's an important point there, because the best thing i like about agile is that you are working so closely with the user and i've done this before, i confess.
when i was doing a traditional project, i would build a product, go away for a while, come back, present it to the customer, and sometimes, they were happy, and sometimes, they weren't, and you certainly knew when they weren't happy, so that's why i like agile. at least you know those little course corrections. ashik: exactly, so the big success story we have, as an example, at the bbg, we are building mobile applications, bbg ? voice of america is the leading network, but in the bbg, they put our audio, video, and there are articles for a global audience, and this content,
we are trying to reach to a much younger demographic across over 100 countries in 200 different languages, and these, voice of america news and other applications that you have built have become hugely popular with this target audience of younger demographic and via one or be a finalist in many awards, so that's a big success story i can cite. in addition, we are also building some api's for people to consume all this content in a much more easier way, so these are the practical success stories i can cite for the agile process of
the pu's. bill: great, so what are three things you want everyone here to remember at the end of this talk? i keep putting you on the spot. ashik: not a problem. agile gives you two big things, one is agile teams have much higher morale in practice day to day, until they are much more productive,
so you're getting higher value out of the team, and two, you are also minimizing risk, because at the end of every two weeks, you got working code. at any point, you can say, "all right. that's enough and i've got a working product, i can use it today," so you're minimizing risk, maximizing productivity, so in short, you are getting the most software bang for the taxpayer buck. what better can there be? bill: right, the thing i like about it is you got happier customers,
at the end of agile projects. all right, thank you all. ashik: thank you. speaker 5: then, there was one, so high-energy panel. we're actually ahead of schedule, which is awesome. at the end, you never can really say that too much, so i'd like to thank everyone who has spoken. i'd like to remind you if you have to cut out a little early please fill out the survey that's in the back of your id badge here, but next up,
we're going to be talking about customer experience and opening up the customer experience, and we have three awesome agencies to talk about that and three awesome people, andrew huey from [audio distortion] talking and moderating, stephanie and david from xm bank and my uscs project, respectively. andrew, take it away. andrew: thanks, jacob. last panel, it's been a nice session today. i really enjoyed it.
stephanie, david, thanks for joining us. stephanie: thank you for having us. andrew: we've heard a lot of stuff about open ways to drive technology, open ways to drive the user experience, and i think we're going to spend a few minutes talking about taking some of these either internal lessons or user experience lessons focused on specific projects, and how do we take that across the whole organization to focus on what our customers, whoever they might be, are really looking for.
at the irs, we're doing all kinds of things to create a consistent customer journey, and it's not simple it's not easy, we're doing usability testing. we're doing journey mapping. we're doing focus groups. we're doing interviews with taxpayers, with tax preparers, contextual reviews. we're watching our app store reviews and minding those for just the common snapshot reviews, which can be really bad sometimes,
but there's usually a kernel of an idea of something, "hey, we should, maybe, look into that. how does that fit into our larger picture?" one project the team's working on right now is there is a set of forms people ? a set of forms at the irs, go figure. there's a set of forms people send in to, basically, apply for a certain kind of categorization, and we ask for some specific documents with that. well, they send those documents and then a bunch more,
so it was like an income verification document, or a business sass document. they don't send in a business license, or their utility bill proving that they do live in that building. we don't care about that. we just want a couple of documents. internally, our process have been, "well, whatever they send us, we have to look at we have to tag. we have to manage the file," and that creates a huge burden on our internal folks doing that,
so we need to change our policies to say, "maybe we don't have to tag and file the irrelevant stuff. we can mark that as not important, but then someone starts looking at the process, and like, "well, why are people sending us these irrelevant documents anyway? what do we need these for?" traditionally, it doesn't matter. we'll just tag them, ignore them. now, let's go talk to the actual people who sent those forms in and figure out what's happening.
why do you think we need these? what's going through your minds? what can we do to make the process clearer, and also, save you less work? you're saving the minutes. it's taking time to pull them together. you're probably a little nervous, concerned, frustrated like, "let's send everything in. it's the irs. they're scary," so we've been trying to figure out more
and more ways to take a customer-driven approach, "what is it that our taxpayers, our tax repairers, our third party partners need from us to better either serve their customers or save time, money and frustration?" part of this whole process, things i didn't like, "who is our customer? how do we identify them? how do we work with them?" i think i want to hear from stephanie and david, "how do you guys do it? how do you take this customer-driven approach?"
stephanie: well, the way that we figure out who our customers are as we take, and have since the beginning--i've been with the bank for about two and a half years and this is my sole focus, is on customer strategy for our agency. we have some really great foundational documents, our charter and our congressional mandates really help us to identify who our customers are. those would be united states exporters, and because we also rely very heavily on intermediaries like financial services institutions and insurance brokers.
we regard our intermediaries as customers as well, because we need them to reach our mandates and to reach our end user customers. andrew: thanks, david? david: i think i'll actually move down, because i am a little bit scared that andrew is up here. with uscis, we have a very unique mission. we started way back in 1896, and the way it started, of course, is that many people, when they came into the united states, the very first thing that they saw was the statue of liberty,
and it kind of evoked this sense of spirit and belonging and welcoming to the united states. well, how do you do that in the digital era? we were fortunate enough to have plenty of demographics about our customers. we have studies done by independent organizations. we have studies done by ourselves, but demographics and quantitative numbers don't really tell the true story. going back to many of the things that andrew said, we also have been engaging particularly with what we've been doing with my
uscis, it's taking a more user-centered approach, interviewing customers, understanding what their needs are, what their goals are, what motivates them, and then weaving that into the way we design our services. for instance, with my uscis, one of the key things that we've done is introduce a responsive design. prior to this we did not have responsive design, and my colleague who is in the audience, mister jeffrey. maybe he's going to join us from the epa. he's been bringing in some ideas for how we could do this for uscis overall,
but thinking about what we're doing with uscis, why we're doing what we're responsible for, roughly, 30 some odd percent of our customer base are hispanic. a majority of those folks do go on the internet, but they don't use ? the gentleman with the dell laptop in front of us, they don't use dell laptops. they don't use macbooks. they use smart phones, they use flip phones, so global mobile response was very important, so one of the key things for us is really understanding who the users are
and the characteristics, and not just the demographic. stephanie: one of the things that i really like about what you just said was that it's not all about surveys, that is has to branch into other mechanisms, sometimes, like interviews and roundtables and i know that the ux community that gsa sponsors, talks about focus groups and round tables and interviewing, and i think that that's really important in customer experience as well, because when one mechanism works for one particular customer segment, it's not necessarily going to work for another.
if you truly want to engage them and get their feedback, you have to meet them, how they want to be met. andrew: thinking about the data we already have on our customers, especially with immigration and in the irs, we've got a lot of data. we know a lot about you, but we want to be able to use it in a way that's not creepy, and if we just rely on raw numbers that's hard, so we need to talk and engage. stephanie: yeah, there's such a fine line nowadays with respect to what's beneficial
and what's creepy, and it's very difficult in the customer experience community and governing, as well as in the private sector, to get an agreement on where that line is. david: yeah, i think that's very important when you consider the emergence of social media as one way in which we could communicate with our customers. we don't want to give the impression that we are big brother, even though we are to an extent, but the social media realm is one area where i think we can really see some
dramatic change that we can make as government organizations in terms of meeting our customers where they are. stephanie: when we're talking about customer experience within the government setting as well, i know this was one of the things that we wanted to talk about today is it's important not just to have those external metrics and what the customer's experience is, but you have to balance the realities of being in the organization that you're in, so you have to make good use of that internal data that you have as well.
for us at xm bank, we take a very strict view of processing time, because our customers have told us that that's very important, so we have those internal measurements where we measure processing time based on the product line, and we balance that with the external metrics so that we can bring a holistic and authoritative view of our customers and our partners to the table. david: sometimes, i think measuring success, i know there's a lot of discussion kpi's earlier, is well, we got to define early on, "what is success?
what are our goals?" and then let's start tracking them. do you find it helpful to share that? maybe you're sharing that publicly, or you're sharing that up and down the management chain? who gets to see that data? stephanie: right, so with respect to processing time we have a few lenses within xm bank. first of all, our employees are able to see processing time instantly any time they want to.
they can log in... david: everyone. stephanie: everyone. everyone has access to this data. i have an intranet site that i built at xm bank, there's a link on my intranet page where an employee can log into our system. it can get there a lot of other ways as well, but if they'd like to log in through my page, they can, and they can get a view of what the processing times are on average, currently,
for completed transactions within the bank. we also have a weekly operations review committee meeting. this is something that's really important as well, because it's brought a rhythm and cadence to the data review to the senior-most levels of the organization as to how long these transactions are taking to be processed, and then additionally, for transactions that have been complete, i present to senior leadership. i'm part of the chairman hochberg senior team.
i present to the chairman every week, as well as to the rest of the senior team, a view, a snapshot of the average processing times for the highest volume products within the bank. now, publicly we also report--this as part of our report every year--for the high volume product lines, we will publish the number of cases as well as the average processing time, because we do want that public accountability and that transparency.
andrew: what do you think would happen if you made the live internal numbers public on a daily basis? stephanie: that's a really good question. i would not have a problem with that. we want our customers to understand how long it's going to take. we want our customers to understand what to expect. we would like for their experiences to match their expectations. we are going to be doing more frequent publishing of that information. we haven't thought about doing it daily,
but that is not something that i would shy away from, in any way shape or form. we want our customers to know. andrew: i'm just thinking the new gap page analytics page for all the government,a lot of people were really curious how many people are visiting government websites, beyond just our internal stakeholders. david, what about you, how are you measuring success? david: actually, with my uscis, one of the most ? i think we're doing a lot of experimental things, so we do have the typical metrics to number of people that hit the site,
but we also want to incorporate principles like lean startup. what i mean by lean start-up, we try to measure the most critical few metrics that really matter to our customers and how that affects their overall experience, so for instance, we try to monitor these particular things that we set up based on the way we designed our systems to try to ensure that customers follow a certain user flow. well, if they're not following that flow and we have metrics measuring that, if they're not clicking on a particular area of our site, then we understand that we actually have to adjust and change.
going back to one of the early points that stephanie was bringing up, i do think one of the key things is, really, transparency in government, so within uscis, we do our best to be very transparent about our processing times. of course, we're called to the hill many days, and we do share our data with our customers. i think we are now in the place with the convergence of both technologies. under this administration with the open.gov movement, we are now in a place where we're starting to be able to expose more
information than we ever have before. as a result, we have a four-year request, but we do our best in order to share the metrics, but more importantly, i think it's important to start measuring what matters to the customer and to the organization to meet those goals. stephanie: i think that that's so important, the metrics that matter, because when if you're just starting a program or just trying to get a strategic program in place ? and there's a book called, "lean analytics," and it talks about that metric that matters,
and i think that it's a really great book. when you're just starting out and building a program, it's really important to hit those metrics that matter, and sometimes, you can have a difficult time trying to figure out what are those metrics, but once you land on them, it's a place to start. andrew: i'm glad you mentioned starting out and how that's difficult, making sure you have the right strategy in capturing the right metrics. there is kind of a larger question, though: why even bother? for the googles and the facebooks of the world,
it makes more sense why they have this real customer focus. xm bank, irs, homeland security, these do not inspire fans of customers. they're not, "yeah, i can't wait to get my next irs notice. " why are we exposing ourselves like this? what does it gain, either a customer, or more importantly, us? stephanie: i joined government from 24 years in the private sector, previously, and i did customer experience work for bernstein young, as well as in the glow, and it just makes fundamental business sense to include your customers because you can gain efficiencies, and you can figure out how to lean your processes,
for example, and that's mutually beneficial. if you're bringing on this entire discipline, management discipline of customer experience, you can figure out ways to improve your business and the bottom line as well. specifically, with respect to xm bank, we are a very unique government agency in that we actually do have competitors. can you believe that? there is a government agency out there that does have competitors. we compete with other export credit agencies around the world
and we're mandated to be competitive. our customers do have a choice in some instances, so we want to make sure that we're capturing their voice and bringing it to the table in the most effective way possible. david: why it matters for uscis? i think the biggest point is that we deal with human beings. we deal with people that may be coming from oppression. they may be coming from very desperate economic conditions, and why it matters is because these are people that have hopes.
they have dreams, and being able to provide them with the level of service that really shows that the united states is what it's meant to be, going back to emma lazarus's poem about welcoming your masses. that's why it matters because the customers are humans, and we recognize that they come from all sorts of different socio-economic conditions and they come here for different reasons, and those, not just measuring just the metrics but also making sure that the experience itself is the right experience for them that that experience echoes the
sentiment welcome is really why it matters within uscis. of course, i did want what stephanie said. it doesn't matter to our bottom line because we are, we do have competitors. we do represent the united states when you think about it. everyone is competing for immigrant talent in the major economies. bringing in the right immigrant mix here helps continue to build on the legacy that many immigrants before built for the united states. andrew: one thing about customer experience is it tends to be a little more strategic than some of the more tactical approaches.
we're taking time to think it through and make sure we cover a lot of different functions within our organizations no matter how our customers are interacting with us. has this forced any internal changes or internal assessments within your agencies? stephanie: as far as internal changes, absolutely. i think that there is no way to have this discipline, this new management discipline injected into an organization and there not be internal changes.
i think that that's a really great place to begin looking, because communication is fundamental to everything in customer experience, period to the extent that we hear from our customers that we're not doing a good job at communicating internally as [audio distortion] the siloes, then we need to do something about that. that is something that is on our minds, and that has been a really big change for us, and it has challenged the way that we communicate internally and helped us to understand the changes that we need to make
and the improvements that we need to make as a team. david: change is required, and i think the biggest change is the cultural change. we went from a kind of a manufacturing mindset. we have this many millions of cases that we receive every year. we have this many that have to get out. we have congressional mandates that we have to hit these certain numbers within this set time. well, that's well and good,
but the most important change that needs to happen in there is the cultural change, so changing from the perspective of an inside-out orientation was the biggest change for us. we had to literally change it to an outside-in orientation. we had to really make folks understand that "hey, these are the people that we serve. these are the people that are coming to use our services," so we have to start from them in order to really make the types of improvements
and change that you want. stephanie: i think that it can be extremely difficult in government. i see we only have a few seconds left, but it can be extremely difficult in government, because we do have to balance the reality of being government organizations with the expectations of our customers, and sometimes, we can't have the kinds of conversations with customers where they're going to walk away being completely satisfied, so we have to have a balance. andrew: real quick, for the time we have left,
this has all been driven by customers, driven by feedback, where does it go from there? is it top-down from mandarin's vision? is it bottom up from people doing the interaction making those changes? stephanie: cross disciplinary for us across, no matter how you slice it, taking people from the top, the bottom, the sides, all across the organization. david: yeah,and for us it's definitely have been driving this from the bottom-up for us because it starts with the customer, and the next level was that front line person that deals with the customer
and taking their perspectives, because they're the closest thing that we have internally to someone that really has the voice of the customer. andrew: thanks, david. i just want to share a quick anecdote of this sort of working across siloes, and stephanie and i were talking about earlier, she was saying she had, probably, she was hoping he could be here. dave is watching the live stream. stephanie: our contacts and our manager rochelle barroom, and she
and i were talking about this today and if she's watching, hello, rochelle. andrew: you're not living in digital ghetto. you're actually putting people together to get the words out. stephanie: yes, we actually are. thank you for banging on the wall next to you and get to someone. andrew: that's so great to hear. thank you, stephanie.thank you, david. andrew: thank you everybody. speaker 5: thank you david, stephanie and andrew.
that was a great way to end the day. i would like to personally thank bernetta reese who is doing the live stream. let's give her a big round of applause. bernetta, thank you. we've had so many ideas for interviews and just watching you out there doing the expo was just amazing and you kept focused. they were all these crazy great things going on and you're on it. thank you so much.
as i mentioned, survey, back here for those of you in person, we will be sending out a bigger survey to everyone after the event, but we'd like to get some initial feedback. that really wraps up our event. i hope you enjoyed it.we will be posting the slides and an archived version of the whole event. i am told early next week, so be on the lookout for that. david bright just tweeted out to say that he was going to be putting his slides on slide share later today so we take a look at it, fcccio on twitter. he'll probably tweet out when he does that,
if you're into that internet of everything. keep an eye on digitalgov. i will be doing little targeted pieces on some of the content you saw today and some of the presentations you did. all the presenters were great. when i counted lasts night between all the stuff in the room and all the stuff out of the expo, we had 30 different agencies in the federal government represented. i think it's really good.
it's really amazing. last point, do the survey, please hand your badges out to some folks who'll be collecting as you leave this way, and keep on keeping on.
