Through September I ran a survey to establish UK Executive's attitudes towards custom Software Development. Unfortunately the survey failed to obtain enough respondents to be useful, but I want to share my experience. So in this podcast, I largely cover how the survey was designed, how I promoted it and the lessons learnt.
Or listen at:
Published: Wed, 04 Nov 2020 17:26:49 GMT
Transcript
[00:00:35] Hello and welcome.
[00:00:37] During September, I started a survey. The survey was aimed at UK executives to try to understand their attitudes towards custom software development.
[00:00:49] I ran the survey because I believe in three things: one - every business is a technology business, two - businesses are becoming more reliant on custom software development just to keep pace, and three - traditional management techniques don't work well for custom software development.
[00:01:12] The survey was intended to test those beliefs.
[00:01:17] Unfortunately, I only got a single response. As such, I can't really talk about results - simply because I don't have them for the survey, which is a shame. So this episode will be predominantly focused on what I actually did, how the survey worked and the steps I took and some of the lessons learnt.
[00:01:38] Unfortunately, I won't be talking about any results that I've been able to gain from the survey; so feel free to skip this episode if you're not interested in the mechanics of how I was actually doing it. Thank you.
[00:01:53] So the survey I went for was a fairly simple five page survey. I wanted to keep it nice and simple and short to encourage respondents (obviously that didn't work, but that's a separate problem).
[00:02:06] So in the first page, I looked at demographics, I wanted to capture what sort of size of an organisation and seniority the respondent was, this, I felt, would help me understand whether or not there was a difference between the size of the organisation in terms of the responses and where in that organisation structure they were. Potentially also to understand, if they provided their job title, as to whether or not there was a different view, depending on whether you worked within I.T. Directly, potentially a CIO or CTO versus outside of I.T., such as a CEO or CFO.
[00:02:44] The second page was used to understand what software types in use and how critical they were to an organisation. So there's lots of different types of software you can get now. I listed six and an other in the survey.
[00:02:57] So I had e-commerce, which would be a website selling products or services. Information website, a website providing information only something like a portfolio or a blog, etc. Mobile apps, applications that could be installed on a mobile or tablet. Line of business applications, applications that are intended for running internal to your organisation for running business specific processes. Software as a service, a product that you're actually building to sell out as a service, normally in a subscription model. And then just a category of "Other" - for anything that didn't really fit into one of those other categories. This was just to get an idea of how critical and how widespread software development was within in the respondents.
[00:03:49] The third page was about getting the respondents experiences; understanding how they felt. So this was the cusp of trying to get their attitudes towards software development across seven categories.
[00:04:04] The first of those was ROI - did the respondent feel that they were getting a valuable return on investment. I allowed them to grade that between a value of one to five, one being not very, five being very. And obviously this is very much an individual's own response, as is true of any of these categories.
[00:04:26] The next was keeping pace - is it keeping pace with the business needs?
[00:04:32] Then went on to recruitment - is it easy to recruit the developers you want? Retention - is it easy to retain your developers?
[00:04:40] Then I talked about quality and predictability.
[00:04:44] Again, all of those seven categories asking the question, how happy does the respondent feel about each of those measuring them between one and five.
[00:04:55] On the fourth page, I asked a simple question - what's the one thing you change about your customs software development? I allowed this to be free text to allow anyone to enter exactly what they wanted. It was optional as well. So if they didn't have anything they particularly wanted to answer, they could skip it.
[00:05:12] But I was looking forward to getting some interesting responses to that because I think that could have driven some really interesting conversations. But fortunately, as I've said, the lack of respondents probably makes that moot.
[00:05:26] And then the fifth page, the final page, just asked whether or not the respondent would like to have further contact, whether that was for the results of the survey, any follow up questions or any future surveys.
[00:05:40] And that summarises the survey. So the survey itself could be done probably in two or three minutes. I believe I've got the right size to allow people to jump on, do it quickly within their working day.
[00:05:54] So let's talk about promotion.
[00:05:57] So obviously, my main problem appears to have been to actually get respondents to the survey to actually take it in the first place. Now, when I ran this I ran this through September, I tried a number of things to try and promote it.
[00:06:10] First of all, was the obvious one using LinkedIn and just posting to my regular network. I did a couple of extra bits there in terms of including a PowerPoint slide show effect. So there's an interesting thing that you can do in LinkedIn; if you share a document, it will actually make like a little slideshow and that, when it shows in the timeline, you can click through it.
[00:06:33] So that seemed quite an interesting one and starting to learn from this process.
[00:06:39] I also ran a carousel ad within LinkedIn. And this is a paid advertisement rather than just on my personal network. That was using their carousel ad time, which allowed you to set up a slide show. So similar to the PowerPoint I've done in my own network. It allowed me to put four slides in.
[00:07:02] It got a reasonable number of impressions. It was just over 2000 impressions, but I didn't get any responses from it. And I've got to admit, I've always struggled to get much response from LinkedIn advertising. I do question whether or not I'm using it right or whether the actual return is there in the first place. I suspect given almost the niche quality of the respondents I'm looking for, that it was difficult to target down just using something as blunt as a carousel ad. After I'd run the carousel ad for a period of time, I then moved on to using message ads in LinkedIn. So the email versions.
[00:07:42] These are a bit more ,for me, I feel a bit more personal because they're going straight into your inbox. They are text only predominantly. Certainly the ones I did were text only and just feel a lot more personable. I certainly probably react more to emails coming in to my LinkedIn mailbox than I do to seeing things in the feed.
[00:08:06] That one got about 9000 I'm sorry, 900 impressions, but I did get a response from that. And that was the only response I got, and unfortunately, it's definitely isn't a great return. And again, I've struggled to get return on any advertising type I've used with LinkedIn. So unfortunately, those didn't really work the way I wanted them to.
[00:08:28] But again, I suspect it is because I've not quite got it right in terms of how I'm targeting the right people and then incentivising them to take time out of their day to actually answer the question.
[00:08:41] I've used Google ads before and I chose not to use it for this. So I've used them in the past for podcasts. And while it will drive a lot of traffic, I found the traffic to be quite poor. So I certainly didn't want to try and do this because I thought, I'm going to do is get a lot of noise. So I chose not to do this. When I ran it for the podcasts, I certainly saw a lot of traffic going into the website to listen to the podcast, but it wasn't sticky traffic. It wasn't qualified, it wasn't quality traffic, unfortunately.
[00:09:12] So I've got to admit, I've struggled to use that ever to actually get good value.
[00:09:19] The last thing I want to talk about is Serve Survey Circle. So Service Circle is a website where you post your survey, and the idea is that all the members of Survey Circle then basically game-ify between the the surveys to effectively do each other's to try and help each other gain respondents.
[00:09:41] Now, my case, because I'm probably quite narrow focused in the respondents I wanted, I'm looking for UK executives, then I got zero responses from that. If I'm honest, its probably more of an academic place. So unfortunately, no responses from there either.
[00:09:59] So lessons learnt.
[00:10:03] I've got to admit, I've actually got quite a bit I have learnt from doing this survey, even if I haven't actually got any results to look at and talk about.
[00:10:11] So technical lessons. I actually used a whole set of new technologies in terms of building this survey. So I used a system called Blazer, which is a new Microsoft method of developing websites. So something that's been on my to-do list for probably 6 to 12 months now. So produced a really good excuse to do that. I'm actually blogging separately on my experiences with that. It looks like it's quite a new technology, a little bit early in its development for most production uses, but it allows me a good excuse to get my teeth into it, understand it, understand where Microsoft are going with it. It being a Microsoft product, it looks like it's going to be core to their web development for potentially the next 5, 10, 20 years - who knows? But certainly one of their key products at the moment. So if nothing else, just doing that has been a great learning exercise and a good investment of my time.
[00:11:13] While my marketing didn't go very well, it has increased my experience, and so I've experienced a number of new types of marketing campaign with LinkedIn.
[00:11:25] So whether it be the Carousel ad, which I haven't done previously, or the message ads, again hadn't done that previously or even just the sharing of documents within my own timeline, that wasn't something I'd done before. So again, all of these were quite useful just to gain experience. I'm still not there in terms of being confident in terms of how to run a marketing campaign that is going to yield the results, I would like, but certainly. Again, more experience, more knowledge, more information to take forward.
[00:11:58] And the other lesson obviously learnt is I'm struggling to find the right people to talk to. It's a bit of a difficult community to get to. And the whole of this podcast and many of the things I do are actually aimed at people that don't know they have a problem. I'm actually trying to talk to UK executives that don't know that, they don't know, how to get the best out of software development?
[00:12:23] Which can be quite a difficult thing. I'm trying to reach out to people and find people who at the moment may be oblivious to the problem they have.
[00:12:33] And a lot of what I do, is try to highlight where things don't work that way. So I'll be looking at a number of things over the course of probably the next 6 to 12 months to look at how to build that community and try to see whether I can find a way of advertising in trying to find like-minded people.
[00:12:53] One of those actions I've done recently is I've started a LinkedIn group to do exactly that. I'm going to promote it probably some point in November as a means of being getting effectively free consultancy for managing software development.
[00:13:14] I'd like to turn it into an environment where the community can grow for those people that feel they're struggling to get the best out of their software development ROI - and they can come there and get answers.
[00:13:26] But I'll talk about that more as and when I get time to run that properly and try to promote out. I hope to, as I say, to turn it into a community with very much a question and answer based feel rather than just being another one of those groups that's just constantly getting adverts posted to them.
[00:13:46] So it's very much a shame about the number of responses I got from the survey. It's quite disappointing. I'd have liked to obviously have got more, but maybe in the future I shall try and rerun it again once I've got the right level of community available to me.
[00:14:04] And thank you for listening to this slightly different podcast. This one's obviously not been specifically about how to improve ROI, but probably just a bit more of a look under the covers of some of the work that I'm trying to do to advertise my work out there to try and bring what I believe are critical things to the wider audience.
[00:14:26] Thank you for listening. I look forward to speaking to you again next week.